2024-01-20T08:14:37.427 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-20T08:14:37.428 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} --archive /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433 --name yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/orig.config.yaml 2024-01-20T08:14:37.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-20T08:14:37.550 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433 branch: main description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} email: yweinste@redhat.com first_in_suite: false job_id: '7522433' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mds beacon grace: 60 mds heartbeat grace: 60 osd heartbeat grace: 60 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds valgrind exit: true mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd crush smoke test: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow request - SLOW_OPS - MON_DOWN sha1: a00c9940991c5e1e9ab595faa3f6e1281d8ae167 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes watchdog: daemon_restart: normal ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: valgrind: - --tool=memcheck - --leak-check=full - --show-reachable=yes check-counter: counters: mds: - mds.exported - mds.imported install: ceph: debuginfo: true flavor: default sha1: a00c9940991c5e1e9ab595faa3f6e1281d8ae167 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: a00c9940991c5e1e9ab595faa3f6e1281d8ae167 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 450 sha1: a00c9940991c5e1e9ab595faa3f6e1281d8ae167 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: a00c9940991c5e1e9ab595faa3f6e1281d8ae167 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGB32BTEDO0f4g3p4Su/WnQlZgOmLrecAJhuA4qWv3Hx1cNDmUiiybUgT27c7A5Yy4fY0ICXZmnZzUdRNPO3Oio= smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE16rhgvJXoLMH67byLqrLa9Gu1eNFmFGCvaBgtCERHmUQ1R4dGsq0qngj1/LXWBVgR6VFpcqXa3CROxwDuOR1k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-19_15:24:44 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-20T08:14:37.550 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa; will attempt to use it 2024-01-20T08:14:37.551 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks 2024-01-20T08:14:37.551 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-20T08:14:37.552 INFO:teuthology.task.internal:Checking packages... 2024-01-20T08:14:37.578 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167' 2024-01-20T08:14:37.578 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-20T08:14:37.578 INFO:teuthology.packaging:ref: None 2024-01-20T08:14:37.578 INFO:teuthology.packaging:tag: None 2024-01-20T08:14:37.578 INFO:teuthology.packaging:branch: main 2024-01-20T08:14:37.578 INFO:teuthology.packaging:sha1: a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:14:37.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-01-20T08:14:37.746 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-772.gd7e2a322 2024-01-20T08:14:37.747 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-20T08:14:37.754 INFO:teuthology.task.internal:no buildpackages task found 2024-01-20T08:14:37.754 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-20T08:14:37.771 INFO:teuthology.task.internal:Saving configuration 2024-01-20T08:14:37.787 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-20T08:14:37.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-20T08:14:37.818 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-20 08:09:25.502947', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGB32BTEDO0f4g3p4Su/WnQlZgOmLrecAJhuA4qWv3Hx1cNDmUiiybUgT27c7A5Yy4fY0ICXZmnZzUdRNPO3Oio='} 2024-01-20T08:14:37.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-20 08:09:25.501498', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE16rhgvJXoLMH67byLqrLa9Gu1eNFmFGCvaBgtCERHmUQ1R4dGsq0qngj1/LXWBVgR6VFpcqXa3CROxwDuOR1k='} 2024-01-20T08:14:37.837 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-20T08:14:37.843 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-01-20T08:14:37.843 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-01-20T08:14:37.843 INFO:teuthology.run_tasks:Running task console_log... 2024-01-20T08:14:37.914 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f48e30ec430>, signals=[15]) 2024-01-20T08:14:37.914 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-20T08:14:37.920 INFO:teuthology.task.internal:Opening connections... 2024-01-20T08:14:37.920 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:14:37.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:14:38.011 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T08:14:38.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:14:38.120 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-20T08:14:38.127 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-01-20T08:14:38.144 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-01-20T08:14:38.144 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-20T08:14:38.201 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2024-01-20T08:14:38.202 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-20T08:14:38.202 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-20T08:14:38.202 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-20T08:14:38.202 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-01-20T08:14:38.225 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-01-20T08:14:38.242 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-01-20T08:14:38.242 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="9" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="9" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el9" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2024-01-20T08:14:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:LOGO="fedora-logo-icon" 2024-01-20T08:14:38.300 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-20T08:14:38.300 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2024-01-20T08:14:38.300 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-20T08:14:38.300 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-20T08:14:38.300 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-20T08:14:38.300 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-01-20T08:14:38.317 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-20T08:14:38.326 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-20T08:14:38.331 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-20T08:14:38.331 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-01-20T08:14:38.334 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-01-20T08:14:38.354 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-20T08:14:38.360 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-20T08:14:38.360 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-01-20T08:14:38.390 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-01-20T08:14:38.418 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-20T08:14:38.471 INFO:teuthology.run_tasks:Running task kernel... 2024-01-20T08:14:38.485 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-01-20T08:14:38.485 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-01-20T08:14:38.485 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-20T08:14:38.486 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-20T08:14:38.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T08:14:38.502 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-01-20T08:14:38.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T08:14:38.502 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-01-20T08:14:38.558 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-361.el9.x86_64 2024-01-20T08:14:38.558 INFO:teuthology.task.kernel:Running kernel on smithi116: 5.14.0-361.el9.x86_64 2024-01-20T08:14:38.558 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum install -y kernel 2024-01-20T08:14:38.560 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-361.el9.x86_64 2024-01-20T08:14:38.560 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.14.0-361.el9.x86_64 2024-01-20T08:14:38.560 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2024-01-20T08:14:40.770 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2024-01-20T08:14:40.818 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2024-01-20T08:14:42.127 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-01-20T08:14:42.445 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-01-20T08:14:43.244 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-01-20T08:14:44.184 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-01-20T08:14:48.510 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 19 MB 00:03 2024-01-20T08:14:53.993 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 5.9 MB 00:02 2024-01-20T08:14:55.328 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 15 kB 00:00 2024-01-20T08:14:58.271 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 20 MB 00:02 2024-01-20T08:15:01.244 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-01-20T08:15:03.045 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-01-20T08:15:05.143 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-20T08:15:05.144 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-20T08:15:05.246 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout: kernel x86_64 5.14.0-409.el9 CentOS-BaseOS 6.0 M 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core x86_64 5.14.0-409.el9 CentOS-BaseOS 21 M 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules x86_64 5.14.0-409.el9 CentOS-BaseOS 39 M 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core x86_64 5.14.0-409.el9 CentOS-BaseOS 33 M 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-01-20T08:15:05.258 INFO:teuthology.orchestra.run.smithi116.stdout:Install 4 Packages 2024-01-20T08:15:05.259 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:05.259 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 99 M 2024-01-20T08:15:05.259 INFO:teuthology.orchestra.run.smithi116.stdout:Installed size: 122 M 2024-01-20T08:15:05.259 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-01-20T08:15:05.638 INFO:teuthology.orchestra.run.smithi116.stdout:(1/4): kernel-5.14.0-409.el9.x86_64.rpm 16 MB/s | 6.0 MB 00:00 2024-01-20T08:15:06.088 INFO:teuthology.orchestra.run.smithi116.stdout:(2/4): kernel-core-5.14.0-409.el9.x86_64.rpm 25 MB/s | 21 MB 00:00 2024-01-20T08:15:06.439 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 5.9 MB 00:01 2024-01-20T08:15:06.913 INFO:teuthology.orchestra.run.smithi116.stdout:(3/4): kernel-modules-core-5.14.0-409.el9.x86_6 26 MB/s | 33 MB 00:01 2024-01-20T08:15:07.264 INFO:teuthology.orchestra.run.smithi116.stdout:(4/4): kernel-modules-5.14.0-409.el9.x86_64.rpm 20 MB/s | 39 MB 00:02 2024-01-20T08:15:07.267 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-01-20T08:15:07.267 INFO:teuthology.orchestra.run.smithi116.stdout:Total 49 MB/s | 99 MB 00:02 2024-01-20T08:15:07.267 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-01-20T08:15:07.645 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-01-20T08:15:07.645 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-01-20T08:15:08.488 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 15 kB 00:00 2024-01-20T08:15:08.666 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-01-20T08:15:08.666 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-01-20T08:15:09.807 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2024-01-20T08:15:11.050 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-01-20T08:15:11.698 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-core-5.14.0-409.el9.x86_64 1/4 2024-01-20T08:15:11.830 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-core-5.14.0-409.el9.x86_64 2/4 2024-01-20T08:15:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-409.el9.x86_64 2/4 2024-01-20T08:15:14.713 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-01-20T08:15:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-5.14.0-409.el9.x86_64 3/4 2024-01-20T08:15:16.853 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-20T08:15:16.853 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-20T08:15:16.971 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-01-20T08:15:16.983 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-01-20T08:15:16.983 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 5.14.0-409.el9 CentOS-BaseOS 6.0 M 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 5.14.0-409.el9 CentOS-BaseOS 21 M 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 5.14.0-409.el9 CentOS-BaseOS 39 M 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core x86_64 5.14.0-409.el9 CentOS-BaseOS 33 M 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout:Install 4 Packages 2024-01-20T08:15:16.984 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:15:16.985 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 99 M 2024-01-20T08:15:16.985 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 122 M 2024-01-20T08:15:16.985 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-01-20T08:15:17.456 INFO:teuthology.orchestra.run.smithi087.stdout:(1/4): kernel-5.14.0-409.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-01-20T08:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:(2/4): kernel-core-5.14.0-409.el9.x86_64.rpm 23 MB/s | 21 MB 00:00 2024-01-20T08:15:18.481 INFO:teuthology.orchestra.run.smithi087.stdout:(3/4): kernel-modules-5.14.0-409.el9.x86_64.rpm 26 MB/s | 39 MB 00:01 2024-01-20T08:15:18.940 INFO:teuthology.orchestra.run.smithi087.stdout:(4/4): kernel-modules-core-5.14.0-409.el9.x86_6 23 MB/s | 33 MB 00:01 2024-01-20T08:15:18.943 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-01-20T08:15:18.943 INFO:teuthology.orchestra.run.smithi087.stdout:Total 51 MB/s | 99 MB 00:01 2024-01-20T08:15:18.943 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-01-20T08:15:19.315 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-01-20T08:15:19.315 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-01-20T08:15:19.471 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-409.el9.x86_64 3/4 2024-01-20T08:15:19.645 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:20.329 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-01-20T08:15:20.330 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-01-20T08:15:22.761 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-01-20T08:15:23.426 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-core-5.14.0-409.el9.x86_64 1/4 2024-01-20T08:15:23.557 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-5.14.0-409.el9.x86_64 2/4 2024-01-20T08:15:24.352 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-core-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:26.352 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-409.el9.x86_64 2/4 2024-01-20T08:15:26.536 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-5.14.0-409.el9.x86_64 3/4 2024-01-20T08:15:31.255 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-409.el9.x86_64 3/4 2024-01-20T08:15:31.456 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:36.185 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-core-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:53.705 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:53.705 INFO:teuthology.orchestra.run.smithi116.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-20T08:15:53.705 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:53.775 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:54.149 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:54.149 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-5.14.0-409.el9.x86_64 1/4 2024-01-20T08:15:54.149 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-core-5.14.0-409.el9.x86_64 2/4 2024-01-20T08:15:54.150 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-5.14.0-409.el9.x86_64 3/4 2024-01-20T08:15:54.590 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-core-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-5.14.0-409.el9.x86_64 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core-5.14.0-409.el9.x86_64 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-5.14.0-409.el9.x86_64 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core-5.14.0-409.el9.x86_64 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:54.591 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-01-20T08:15:55.056 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-01-20T08:15:55.203 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-409.el9.x86_64 2024-01-20T08:15:55.203 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-409.el9.x86_64 2024-01-20T08:15:55.203 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-409.el9.x86_64 2024-01-20T08:15:55.203 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-20T08:15:55.203 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-20T08:15:55.204 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-01-20T08:15:55.204 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi116.front.sepia.ceph.com, path=None, version=distro) 2024-01-20T08:15:55.204 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-01-20T08:15:55.311 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-409.el9.x86_64 2024-01-20T08:15:55.311 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-409.el9.x86_64 2024-01-20T08:15:55.311 DEBUG:teuthology.orchestra.run.smithi116:> sudo rpm -qi grub2-tools 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Name : grub2-tools 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Epoch : 1 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Version : 2.06 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Release : 68.el9 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Architecture: x86_64 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Group : Unspecified 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Size : 8264254 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:License : GPLv3+ 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-20T08:15:55.352 INFO:teuthology.orchestra.run.smithi116.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:Packager : builder@centos.org 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:Vendor : CentOS 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:Summary : Support tools for GRUB. 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:Description : 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:hardware devices. 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:15:55.353 INFO:teuthology.orchestra.run.smithi116.stdout:This subpackage provides tools for support of all platforms. 2024-01-20T08:15:55.354 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-20T08:15:55.354 INFO:teuthology.task.kernel:Updating grub on smithi116 to boot 5.14.0-409.el9.x86_64 2024-01-20T08:15:55.354 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-20T08:15:55.966 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-01-20T08:15:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-20T08:15:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-01-20T08:15:57.420 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-01-20T08:15:57.434 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.ngeRuaYXMs 2024-01-20T08:15:57.434 DEBUG:teuthology.orchestra.run.smithi116:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ngeRuaYXMs 2024-01-20T08:15:57.503 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.ngeRuaYXMs 2024-01-20T08:15:57.644 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.ngeRuaYXMs is 6KB 2024-01-20T08:15:57.693 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.ngeRuaYXMs 2024-01-20T08:15:57.707 DEBUG:teuthology.orchestra.run.smithi116:> sudo /bin/ls /boot/loader/entries || true 2024-01-20T08:15:57.772 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-20T08:15:57.773 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-20T08:15:57.773 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-20T08:15:57.773 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-5.14.0-409.el9.x86_64.conf 2024-01-20T08:15:57.774 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-409.el9.x86_64 2024-01-20T08:15:57.931 DEBUG:teuthology.orchestra.run.smithi116:> sudo shutdown -r now 2024-01-20T08:16:05.612 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:16:05.612 INFO:teuthology.orchestra.run.smithi087.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-20T08:16:05.612 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:16:05.682 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:16:06.167 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:16:06.167 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-5.14.0-409.el9.x86_64 1/4 2024-01-20T08:16:06.167 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-5.14.0-409.el9.x86_64 2/4 2024-01-20T08:16:06.167 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-5.14.0-409.el9.x86_64 3/4 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-core-5.14.0-409.el9.x86_64 4/4 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-5.14.0-409.el9.x86_64 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core-5.14.0-409.el9.x86_64 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-5.14.0-409.el9.x86_64 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core-5.14.0-409.el9.x86_64 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:16:06.593 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-01-20T08:16:07.089 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-01-20T08:16:07.230 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-409.el9.x86_64 2024-01-20T08:16:07.230 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-409.el9.x86_64 2024-01-20T08:16:07.231 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-409.el9.x86_64 2024-01-20T08:16:07.231 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-20T08:16:07.231 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-20T08:16:07.231 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-01-20T08:16:07.231 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-01-20T08:16:07.231 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-01-20T08:16:07.338 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-409.el9.x86_64 2024-01-20T08:16:07.338 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-409.el9.x86_64 2024-01-20T08:16:07.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2024-01-20T08:16:07.379 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.06 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 68.el9 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Group : Unspecified 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 8264254 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-20T08:16:07.380 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : builder@centos.org 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:16:07.381 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2024-01-20T08:16:07.382 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-20T08:16:07.382 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 5.14.0-409.el9.x86_64 2024-01-20T08:16:07.382 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-20T08:16:08.001 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-01-20T08:16:09.415 INFO:teuthology.orchestra.run.smithi087.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-20T08:16:09.449 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-01-20T08:16:09.451 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-01-20T08:16:09.466 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.9cNxjL31Nq 2024-01-20T08:16:09.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9cNxjL31Nq 2024-01-20T08:16:09.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.9cNxjL31Nq 2024-01-20T08:16:09.679 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.9cNxjL31Nq is 6KB 2024-01-20T08:16:09.699 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.9cNxjL31Nq 2024-01-20T08:16:09.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2024-01-20T08:16:09.777 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-20T08:16:09.778 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-20T08:16:09.778 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-20T08:16:09.778 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-409.el9.x86_64.conf 2024-01-20T08:16:09.779 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-409.el9.x86_64 2024-01-20T08:16:09.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-01-20T08:16:09.989 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-01-20T08:16:27.950 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-01-20T08:16:27.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:16:27.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:16:39.928 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-20T08:16:39.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:16:39.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:16:46.396 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-01-20T08:16:55.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:16:55.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:16:58.428 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-01-20T08:16:58.460 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-01-20T08:17:07.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:17:07.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:17:10.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:17:10.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:17:10.492 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-01-20T08:17:22.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:17:22.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:17:28.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-01-20T08:17:40.924 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-01-20T08:17:43.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:17:43.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:17:44.421 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-01-20T08:17:44.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:17:44.846 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-409.el9.x86_64"... 2024-01-20T08:17:44.847 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-01-20T08:17:44.860 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-409.el9.x86_64 2024-01-20T08:17:44.861 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-409.el9.x86_64 vs 5.14.0-409.el9.x86_64 2024-01-20T08:17:44.861 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-20T08:17:44.861 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-20T08:17:45.861 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-01-20T08:17:45.862 DEBUG:teuthology.orchestra.run.smithi116:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-20T08:17:45.925 INFO:teuthology.orchestra.run.smithi116.stdout:ttyS1 2024-01-20T08:17:45.968 DEBUG:teuthology.parallel:result is None 2024-01-20T08:17:55.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:17:55.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:17:56.294 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-01-20T08:17:56.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:17:56.705 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-409.el9.x86_64"... 2024-01-20T08:17:56.705 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-01-20T08:17:56.720 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-409.el9.x86_64 2024-01-20T08:17:56.720 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-409.el9.x86_64 vs 5.14.0-409.el9.x86_64 2024-01-20T08:17:56.720 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-20T08:17:56.720 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-20T08:17:57.721 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-20T08:17:57.721 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-20T08:17:57.781 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2024-01-20T08:17:57.818 DEBUG:teuthology.parallel:result is None 2024-01-20T08:17:57.818 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-20T08:17:57.836 INFO:teuthology.task.internal:Creating test directory... 2024-01-20T08:17:57.836 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-20T08:17:57.840 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-20T08:17:57.859 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-20T08:17:57.869 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-20T08:17:57.876 INFO:teuthology.task.internal:Creating archive directory... 2024-01-20T08:17:57.877 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-20T08:17:57.898 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-20T08:17:57.944 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-20T08:17:57.952 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-20T08:17:57.953 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-20T08:17:57.981 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-20T08:17:58.006 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-20T08:17:58.014 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-20T08:17:58.018 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-20T08:17:58.028 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-20T08:17:58.030 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-20T08:17:58.037 INFO:teuthology.task.internal:Configuring sudo... 2024-01-20T08:17:58.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-20T08:17:58.062 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-20T08:17:58.100 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-20T08:17:58.109 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-20T08:17:58.109 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-20T08:17:58.131 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-20T08:17:58.156 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-20T08:17:58.233 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-20T08:17:58.278 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-20T08:17:58.357 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-20T08:17:58.401 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:17:58.401 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-20T08:17:58.468 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-20T08:17:58.509 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-20T08:17:58.558 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-20T08:17:58.640 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-20T08:17:58.684 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:17:58.685 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-20T08:17:58.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-01-20T08:17:58.755 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-01-20T08:17:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-20T08:17:58.835 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-20T08:17:59.257 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-20T08:17:59.265 INFO:teuthology.task.internal:Starting timer... 2024-01-20T08:17:59.265 INFO:teuthology.run_tasks:Running task pcp... 2024-01-20T08:17:59.277 INFO:teuthology.run_tasks:Running task selinux... 2024-01-20T08:17:59.288 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2024-01-20T08:17:59.370 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2024-01-20T08:17:59.373 DEBUG:teuthology.orchestra.run.smithi116:> sudo service auditd rotate 2024-01-20T08:17:59.433 INFO:teuthology.orchestra.run.smithi116.stdout:Rotating logs: 2024-01-20T08:17:59.435 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-20T08:17:59.435 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2024-01-20T08:17:59.466 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2024-01-20T08:17:59.466 DEBUG:teuthology.orchestra.run.smithi116:> /usr/sbin/getenforce 2024-01-20T08:17:59.501 INFO:teuthology.orchestra.run.smithi116.stdout:Permissive 2024-01-20T08:17:59.502 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi087.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi116.front.sepia.ceph.com': 'permissive'} 2024-01-20T08:17:59.502 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-20T08:17:59.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T08:17:59.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-20T08:17:59.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T08:17:59.576 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-20T08:17:59.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2024-01-20T08:17:59.644 DEBUG:teuthology.orchestra.run.smithi116:> sudo /usr/sbin/setenforce permissive 2024-01-20T08:17:59.678 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-20T08:17:59.688 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-01-20T08:17:59.693 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-01-20T08:17:59.807 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-20T08:17:59.827 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-20T08:17:59.829 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi087.front.sepia.ceph.com,smithi116.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-20T08:21:55.224 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi116.front.sepia.ceph.com')] 2024-01-20T08:21:55.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:21:55.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:21:55.313 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-01-20T08:21:55.395 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-01-20T08:21:55.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:21:55.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-20T08:21:55.488 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-01-20T08:21:55.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-01-20T08:21:55.571 INFO:teuthology.run_tasks:Running task clock... 2024-01-20T08:21:55.581 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-20T08:21:55.581 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-20T08:21:55.581 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-20T08:21:55.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-20T08:21:55.584 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-20T08:21:55.612 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-20T08:21:55.629 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-20T08:21:55.640 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-20T08:21:55.657 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-20T08:21:55.658 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2024-01-20T08:21:55.671 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2024-01-20T08:21:55.686 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: ntpd: command not found 2024-01-20T08:21:55.688 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-20T08:21:55.700 INFO:teuthology.orchestra.run.smithi116.stdout:506 Cannot talk to daemon 2024-01-20T08:21:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-20T08:21:55.717 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-20T08:21:55.733 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-20T08:21:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-01-20T08:21:55.761 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-20T08:21:55.761 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-01-20T08:21:55.761 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.761 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.761 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.761 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.786 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-01-20T08:21:55.789 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-20T08:21:55.789 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-01-20T08:21:55.789 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.789 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.789 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.790 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T08:21:55.790 INFO:teuthology.run_tasks:Running task install... 2024-01-20T08:21:55.798 DEBUG:teuthology.task.install:project ceph 2024-01-20T08:21:55.798 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167'}} 2024-01-20T08:21:55.799 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167'} 2024-01-20T08:21:55.799 INFO:teuthology.task.install:Using flavor: default 2024-01-20T08:21:55.804 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2024-01-20T08:21:55.804 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-01-20T08:21:55.805 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167', 'tag': None, 'wait_for_package': False} 2024-01-20T08:21:55.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:21:55.807 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167', 'tag': None, 'wait_for_package': False} 2024-01-20T08:21:55.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:21:55.980 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/ 2024-01-20T08:21:55.980 INFO:teuthology.task.install.rpm:Package version is 19.0.0-763.ga00c9940 2024-01-20T08:21:56.006 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/ 2024-01-20T08:21:56.006 INFO:teuthology.task.install.rpm:Package version is 19.0.0-763.ga00c9940 2024-01-20T08:21:56.122 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-20T08:21:56.122 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:21:56.122 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-20T08:21:56.149 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-20T08:21:56.149 DEBUG:teuthology.orchestra.run.smithi116:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-20T08:21:56.170 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-20T08:21:56.170 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:21:56.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-20T08:21:56.197 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-20T08:21:56.197 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a00c9940991c5e1e9ab595faa3f6e1281d8ae167/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-20T08:21:56.216 DEBUG:teuthology.orchestra.run.smithi116:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-20T08:21:56.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-20T08:21:56.296 DEBUG:teuthology.orchestra.run.smithi116:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-20T08:21:56.344 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-20T08:21:56.363 INFO:teuthology.orchestra.run.smithi116.stdout:check_obsoletes = 1 2024-01-20T08:21:56.364 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum clean all 2024-01-20T08:21:56.410 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2024-01-20T08:21:56.411 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2024-01-20T08:21:56.602 INFO:teuthology.orchestra.run.smithi116.stdout:73 files removed 2024-01-20T08:21:56.629 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-20T08:21:56.677 INFO:teuthology.orchestra.run.smithi087.stdout:73 files removed 2024-01-20T08:21:56.704 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-20T08:21:57.418 INFO:teuthology.orchestra.run.smithi116.stdout:ceph packages for x86_64 169 kB/s | 80 kB 00:00 2024-01-20T08:21:57.426 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 199 kB/s | 80 kB 00:00 2024-01-20T08:21:57.851 INFO:teuthology.orchestra.run.smithi116.stdout:ceph noarch packages 25 kB/s | 9.5 kB 00:00 2024-01-20T08:21:57.868 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 24 kB/s | 9.5 kB 00:00 2024-01-20T08:21:58.194 INFO:teuthology.orchestra.run.smithi116.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-01-20T08:21:58.269 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-01-20T08:21:58.671 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2024-01-20T08:21:58.721 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - AppStream 15 MB/s | 6.4 MB 00:00 2024-01-20T08:22:00.080 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-01-20T08:22:00.088 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-01-20T08:22:00.978 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-01-20T08:22:01.854 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.0 MB 00:01 2024-01-20T08:22:03.563 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-01-20T08:22:05.155 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 19 MB 00:02 2024-01-20T08:22:08.481 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.9 MB 00:01 2024-01-20T08:22:09.697 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 5.9 MB 00:01 2024-01-20T08:22:09.932 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2024-01-20T08:22:11.573 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-01-20T08:22:11.783 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-01-20T08:22:13.685 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2024-01-20T08:22:16.525 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-01-20T08:22:18.491 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-01-20T08:22:19.002 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-20T08:22:19.002 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-20T08:22:19.030 INFO:teuthology.orchestra.run.smithi087.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-20T08:22:19.031 INFO:teuthology.orchestra.run.smithi087.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-20T08:22:19.031 INFO:teuthology.orchestra.run.smithi087.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-20T08:22:19.032 INFO:teuthology.orchestra.run.smithi087.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-20T08:22:19.034 INFO:teuthology.orchestra.run.smithi087.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-20T08:22:19.036 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-20T08:22:19.037 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-20T08:22:19.113 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-01-20T08:22:19.140 INFO:teuthology.orchestra.run.smithi087.stdout:=================================================================================================== 2024-01-20T08:22:19.140 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-01-20T08:22:19.140 INFO:teuthology.orchestra.run.smithi087.stdout:=================================================================================================== 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:19.0.0-763.ga00c9940.el9 ceph 6.6 k 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:19.0.0-763.ga00c9940.el9 ceph 5.1 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 67 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 411 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:19.0.0-763.ga00c9940.el9 ceph 886 k 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 5.9 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-763.ga00c9940.el9 ceph 145 k 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.4 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 48 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.5 M 2024-01-20T08:22:19.141 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 133 k 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 1.7 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 20 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 7.4 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 51 k 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 99 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 286 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:19.0.0-763.ga00c9940.el9 ceph 8.6 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 170 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:19.0.0-763.ga00c9940.el9 ceph 46 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 664 M 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 262 k 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 742 k 2024-01-20T08:22:19.142 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror x86_64 2:19.0.0-763.ga00c9940.el9 ceph 217 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell x86_64 2:19.0.0-763.ga00c9940.el9 ceph 27 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 18 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:19.0.0-763.ga00c9940.el9 ceph 31 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 742 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 5.3 M 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:19.0.0-763.ga00c9940.el9 ceph 127 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 108 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 44 M 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 52 M 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-20T08:22:19.143 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:19.0.0-763.ga00c9940.el9 ceph 162 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 615 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:19.0.0-763.ga00c9940.el9 ceph 320 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 770 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:19.0.0-763.ga00c9940.el9 ceph 297 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.1 M 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:19.0.0-763.ga00c9940.el9 ceph 99 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 337 k 2024-01-20T08:22:19.144 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:19.0.0-763.ga00c9940.el9 ceph 85 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 509 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:19.0.0-763.ga00c9940.el9 ceph 3.1 M 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 52 M 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:19.0.0-763.ga00c9940.el9 ceph 170 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.2 M 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-20T08:22:19.145 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 3.4 M 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 3.1 M 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-20T08:22:19.146 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:19.0.0-763.ga00c9940.el9 ceph 19 M 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 22 M 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 28 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:19.0.0-763.ga00c9940.el9 ceph 2.1 M 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 245 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:19.0.0-763.ga00c9940.el9 ceph 4.7 M 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:19.0.0-763.ga00c9940.el9 ceph 17 M 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 15 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:19.0.0-763.ga00c9940.el9 ceph 25 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-20T08:22:19.147 INFO:teuthology.orchestra.run.smithi087.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:19.0.0-763.ga00c9940.el9 ceph 163 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 499 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 4.8 M 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-20T08:22:19.148 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-20T08:22:19.149 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:19.0.0-763.ga00c9940.el9 ceph 45 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:19.0.0-763.ga00c9940.el9 ceph 130 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-20T08:22:19.150 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-20T08:22:19.151 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-20T08:22:19.152 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-20T08:22:19.153 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout:=================================================================================================== 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout:Install 161 Packages 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 16 Packages 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 2.1 G 2024-01-20T08:22:19.154 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-01-20T08:22:19.717 INFO:teuthology.orchestra.run.smithi087.stdout:(1/177): ceph-19.0.0-763.ga00c9940.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-01-20T08:22:20.443 INFO:teuthology.orchestra.run.smithi087.stdout:(2/177): ceph-base-19.0.0-763.ga00c9940.el9.x86 5.5 MB/s | 5.1 MB 00:00 2024-01-20T08:22:20.971 INFO:teuthology.orchestra.run.smithi116.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-20T08:22:20.972 INFO:teuthology.orchestra.run.smithi116.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-20T08:22:21.006 INFO:teuthology.orchestra.run.smithi116.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-20T08:22:21.007 INFO:teuthology.orchestra.run.smithi116.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-20T08:22:21.008 INFO:teuthology.orchestra.run.smithi116.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-20T08:22:21.009 INFO:teuthology.orchestra.run.smithi116.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-20T08:22:21.011 INFO:teuthology.orchestra.run.smithi116.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-20T08:22:21.014 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-20T08:22:21.015 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-20T08:22:21.093 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout:=================================================================================================== 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout:=================================================================================================== 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph x86_64 2:19.0.0-763.ga00c9940.el9 ceph 6.6 k 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base x86_64 2:19.0.0-763.ga00c9940.el9 ceph 5.1 M 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 67 M 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 411 M 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse x86_64 2:19.0.0-763.ga00c9940.el9 ceph 886 k 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 5.9 M 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-763.ga00c9940.el9 ceph 145 k 2024-01-20T08:22:21.113 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.4 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 48 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.5 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 133 k 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 1.7 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 20 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 7.4 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 51 k 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 99 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 286 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw x86_64 2:19.0.0-763.ga00c9940.el9 ceph 8.6 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 170 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test x86_64 2:19.0.0-763.ga00c9940.el9 ceph 46 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 664 M 2024-01-20T08:22:21.114 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 262 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 742 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror x86_64 2:19.0.0-763.ga00c9940.el9 ceph 217 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell x86_64 2:19.0.0-763.ga00c9940.el9 ceph 27 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 18 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel x86_64 2:19.0.0-763.ga00c9940.el9 ceph 31 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 742 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 5.3 M 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel x86_64 2:19.0.0-763.ga00c9940.el9 ceph 127 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 108 k 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 44 M 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 52 M 2024-01-20T08:22:21.115 INFO:teuthology.orchestra.run.smithi116.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs x86_64 2:19.0.0-763.ga00c9940.el9 ceph 162 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 615 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados x86_64 2:19.0.0-763.ga00c9940.el9 ceph 320 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 770 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd x86_64 2:19.0.0-763.ga00c9940.el9 ceph 297 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.1 M 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw x86_64 2:19.0.0-763.ga00c9940.el9 ceph 99 k 2024-01-20T08:22:21.116 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 337 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse x86_64 2:19.0.0-763.ga00c9940.el9 ceph 85 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 509 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror x86_64 2:19.0.0-763.ga00c9940.el9 ceph 3.1 M 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 52 M 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd x86_64 2:19.0.0-763.ga00c9940.el9 ceph 170 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 1.2 M 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrading: 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-20T08:22:21.117 INFO:teuthology.orchestra.run.smithi116.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: librados2 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 3.4 M 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 3.1 M 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-20T08:22:21.118 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common x86_64 2:19.0.0-763.ga00c9940.el9 ceph 19 M 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-debuginfo x86_64 2:19.0.0-763.ga00c9940.el9 ceph 22 M 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 28 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds x86_64 2:19.0.0-763.ga00c9940.el9 ceph 2.1 M 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 245 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon x86_64 2:19.0.0-763.ga00c9940.el9 ceph 4.7 M 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd x86_64 2:19.0.0-763.ga00c9940.el9 ceph 17 M 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts noarch 2:19.0.0-763.ga00c9940.el9 ceph-noarch 15 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux x86_64 2:19.0.0-763.ga00c9940.el9 ceph 25 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-20T08:22:21.119 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite x86_64 2:19.0.0-763.ga00c9940.el9 ceph 163 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 499 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2 x86_64 2:19.0.0-763.ga00c9940.el9 ceph 4.8 M 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-20T08:22:21.120 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-20T08:22:21.121 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse x86_64 2:19.0.0-763.ga00c9940.el9 ceph 45 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common x86_64 2:19.0.0-763.ga00c9940.el9 ceph 130 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-20T08:22:21.122 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-20T08:22:21.123 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-20T08:22:21.124 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-20T08:22:21.125 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout:=================================================================================================== 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout:Install 161 Packages 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrade 16 Packages 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:22:21.126 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 2.1 G 2024-01-20T08:22:21.127 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-01-20T08:22:21.617 INFO:teuthology.orchestra.run.smithi116.stdout:(1/177): ceph-19.0.0-763.ga00c9940.el9.x86_64.r 18 kB/s | 6.6 kB 00:00 2024-01-20T08:22:22.060 INFO:teuthology.orchestra.run.smithi087.stdout:(3/177): ceph-common-19.0.0-763.ga00c9940.el9.x 8.3 MB/s | 19 MB 00:02 2024-01-20T08:22:22.950 INFO:teuthology.orchestra.run.smithi116.stdout:(2/177): ceph-base-19.0.0-763.ga00c9940.el9.x86 3.0 MB/s | 5.1 MB 00:01 2024-01-20T08:22:25.951 INFO:teuthology.orchestra.run.smithi087.stdout:(4/177): ceph-debuginfo-19.0.0-763.ga00c9940.el 5.7 MB/s | 22 MB 00:03 2024-01-20T08:22:26.218 INFO:teuthology.orchestra.run.smithi087.stdout:(5/177): ceph-fuse-19.0.0-763.ga00c9940.el9.x86 3.2 MB/s | 886 kB 00:00 2024-01-20T08:22:27.268 INFO:teuthology.orchestra.run.smithi087.stdout:(6/177): ceph-fuse-debuginfo-19.0.0-763.ga00c99 5.7 MB/s | 5.9 MB 00:01 2024-01-20T08:22:27.377 INFO:teuthology.orchestra.run.smithi087.stdout:(7/177): ceph-immutable-object-cache-19.0.0-763 1.3 MB/s | 145 kB 00:00 2024-01-20T08:22:27.677 INFO:teuthology.orchestra.run.smithi087.stdout:(8/177): ceph-immutable-object-cache-debuginfo- 4.8 MB/s | 1.4 MB 00:00 2024-01-20T08:22:28.119 INFO:teuthology.orchestra.run.smithi087.stdout:(9/177): ceph-mds-19.0.0-763.ga00c9940.el9.x86_ 4.8 MB/s | 2.1 MB 00:00 2024-01-20T08:22:31.325 INFO:teuthology.orchestra.run.smithi116.stdout:(3/177): ceph-common-19.0.0-763.ga00c9940.el9.x 2.0 MB/s | 19 MB 00:09 2024-01-20T08:22:32.019 INFO:teuthology.orchestra.run.smithi087.stdout:(10/177): ceph-base-debuginfo-19.0.0-763.ga00c9 5.3 MB/s | 67 MB 00:12 2024-01-20T08:22:32.158 INFO:teuthology.orchestra.run.smithi116.stdout:(4/177): ceph-base-debuginfo-19.0.0-763.ga00c99 6.1 MB/s | 67 MB 00:10 2024-01-20T08:22:32.394 INFO:teuthology.orchestra.run.smithi087.stdout:(11/177): ceph-mgr-19.0.0-763.ga00c9940.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-01-20T08:22:32.400 INFO:teuthology.orchestra.run.smithi116.stdout:(5/177): ceph-fuse-19.0.0-763.ga00c9940.el9.x86 3.6 MB/s | 886 kB 00:00 2024-01-20T08:22:33.301 INFO:teuthology.orchestra.run.smithi116.stdout:(6/177): ceph-fuse-debuginfo-19.0.0-763.ga00c99 6.6 MB/s | 5.9 MB 00:00 2024-01-20T08:22:33.401 INFO:teuthology.orchestra.run.smithi116.stdout:(7/177): ceph-immutable-object-cache-19.0.0-763 1.4 MB/s | 145 kB 00:00 2024-01-20T08:22:33.635 INFO:teuthology.orchestra.run.smithi116.stdout:(8/177): ceph-immutable-object-cache-debuginfo- 6.1 MB/s | 1.4 MB 00:00 2024-01-20T08:22:33.943 INFO:teuthology.orchestra.run.smithi116.stdout:(9/177): ceph-mds-19.0.0-763.ga00c9940.el9.x86_ 6.9 MB/s | 2.1 MB 00:00 2024-01-20T08:22:36.653 INFO:teuthology.orchestra.run.smithi087.stdout:(12/177): ceph-mgr-debuginfo-19.0.0-763.ga00c99 4.6 MB/s | 20 MB 00:04 2024-01-20T08:22:37.053 INFO:teuthology.orchestra.run.smithi087.stdout:(13/177): ceph-mds-debuginfo-19.0.0-763.ga00c99 5.3 MB/s | 48 MB 00:08 2024-01-20T08:22:37.611 INFO:teuthology.orchestra.run.smithi087.stdout:(14/177): ceph-mon-19.0.0-763.ga00c9940.el9.x86 4.9 MB/s | 4.7 MB 00:00 2024-01-20T08:22:38.951 INFO:teuthology.orchestra.run.smithi116.stdout:(10/177): ceph-debuginfo-19.0.0-763.ga00c9940.e 2.9 MB/s | 22 MB 00:07 2024-01-20T08:22:39.776 INFO:teuthology.orchestra.run.smithi116.stdout:(11/177): ceph-mgr-19.0.0-763.ga00c9940.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-01-20T08:22:40.136 INFO:teuthology.orchestra.run.smithi087.stdout:(15/177): ceph-osd-19.0.0-763.ga00c9940.el9.x86 6.6 MB/s | 17 MB 00:02 2024-01-20T08:22:40.460 INFO:teuthology.orchestra.run.smithi116.stdout:(12/177): ceph-mds-debuginfo-19.0.0-763.ga00c99 7.3 MB/s | 48 MB 00:06 2024-01-20T08:22:41.443 INFO:teuthology.orchestra.run.smithi116.stdout:(13/177): ceph-mon-19.0.0-763.ga00c9940.el9.x86 4.8 MB/s | 4.7 MB 00:00 2024-01-20T08:22:47.810 INFO:teuthology.orchestra.run.smithi116.stdout:(14/177): ceph-mgr-debuginfo-19.0.0-763.ga00c99 2.4 MB/s | 20 MB 00:08 2024-01-20T08:22:53.643 INFO:teuthology.orchestra.run.smithi116.stdout:(15/177): ceph-osd-19.0.0-763.ga00c9940.el9.x86 2.9 MB/s | 17 MB 00:05 2024-01-20T08:22:54.211 INFO:teuthology.orchestra.run.smithi087.stdout:(16/177): ceph-mon-debuginfo-19.0.0-763.ga00c99 5.7 MB/s | 99 MB 00:17 2024-01-20T08:22:56.494 INFO:teuthology.orchestra.run.smithi087.stdout:(17/177): ceph-radosgw-19.0.0-763.ga00c9940.el9 3.8 MB/s | 8.6 MB 00:02 2024-01-20T08:23:00.826 INFO:teuthology.orchestra.run.smithi116.stdout:(16/177): ceph-mon-debuginfo-19.0.0-763.ga00c99 5.1 MB/s | 99 MB 00:19 2024-01-20T08:23:02.834 INFO:teuthology.orchestra.run.smithi116.stdout:(17/177): ceph-radosgw-19.0.0-763.ga00c9940.el9 4.3 MB/s | 8.6 MB 00:02 2024-01-20T08:23:41.607 INFO:teuthology.orchestra.run.smithi087.stdout:(18/177): ceph-radosgw-debuginfo-19.0.0-763.ga0 3.8 MB/s | 170 MB 00:45 2024-01-20T08:23:41.698 INFO:teuthology.orchestra.run.smithi087.stdout:(19/177): ceph-selinux-19.0.0-763.ga00c9940.el9 275 kB/s | 25 kB 00:00 2024-01-20T08:23:44.921 INFO:teuthology.orchestra.run.smithi116.stdout:(18/177): ceph-radosgw-debuginfo-19.0.0-763.ga0 4.0 MB/s | 170 MB 00:42 2024-01-20T08:23:45.004 INFO:teuthology.orchestra.run.smithi116.stdout:(19/177): ceph-selinux-19.0.0-763.ga00c9940.el9 303 kB/s | 25 kB 00:00 2024-01-20T08:23:49.490 INFO:teuthology.orchestra.run.smithi087.stdout:(20/177): ceph-common-debuginfo-19.0.0-763.ga00 4.6 MB/s | 411 MB 01:29 2024-01-20T08:23:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:(20/177): ceph-test-19.0.0-763.ga00c9940.el9.x8 5.8 MB/s | 46 MB 00:07 2024-01-20T08:23:57.764 INFO:teuthology.orchestra.run.smithi087.stdout:(21/177): ceph-osd-debuginfo-19.0.0-763.ga00c99 3.7 MB/s | 286 MB 01:17 2024-01-20T08:23:57.930 INFO:teuthology.orchestra.run.smithi087.stdout:(22/177): cephfs-mirror-19.0.0-763.ga00c9940.el 1.3 MB/s | 217 kB 00:00 2024-01-20T08:23:57.989 INFO:teuthology.orchestra.run.smithi087.stdout:(23/177): cephfs-shell-19.0.0-763.ga00c9940.el9 465 kB/s | 27 kB 00:00 2024-01-20T08:23:58.056 INFO:teuthology.orchestra.run.smithi087.stdout:(24/177): libcephfs-devel-19.0.0-763.ga00c9940. 468 kB/s | 31 kB 00:00 2024-01-20T08:23:58.087 INFO:teuthology.orchestra.run.smithi116.stdout:(21/177): ceph-osd-debuginfo-19.0.0-763.ga00c99 4.4 MB/s | 286 MB 01:04 2024-01-20T08:23:58.229 INFO:teuthology.orchestra.run.smithi116.stdout:(22/177): cephfs-mirror-19.0.0-763.ga00c9940.el 1.5 MB/s | 217 kB 00:00 2024-01-20T08:23:58.287 INFO:teuthology.orchestra.run.smithi116.stdout:(23/177): cephfs-shell-19.0.0-763.ga00c9940.el9 466 kB/s | 27 kB 00:00 2024-01-20T08:23:58.363 INFO:teuthology.orchestra.run.smithi116.stdout:(24/177): libcephfs-devel-19.0.0-763.ga00c9940. 416 kB/s | 31 kB 00:00 2024-01-20T08:23:58.521 INFO:teuthology.orchestra.run.smithi116.stdout:(25/177): libcephfs2-19.0.0-763.ga00c9940.el9.x 4.6 MB/s | 742 kB 00:00 2024-01-20T08:23:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:(25/177): ceph-test-19.0.0-763.ga00c9940.el9.x8 2.7 MB/s | 46 MB 00:16 2024-01-20T08:23:58.565 INFO:teuthology.orchestra.run.smithi087.stdout:(26/177): libcephfs2-19.0.0-763.ga00c9940.el9.x 1.4 MB/s | 742 kB 00:00 2024-01-20T08:23:58.690 INFO:teuthology.orchestra.run.smithi087.stdout:(27/177): libcephsqlite-19.0.0-763.ga00c9940.el 1.3 MB/s | 163 kB 00:00 2024-01-20T08:23:58.816 INFO:teuthology.orchestra.run.smithi087.stdout:(28/177): librados-devel-19.0.0-763.ga00c9940.e 1.0 MB/s | 127 kB 00:00 2024-01-20T08:23:58.924 INFO:teuthology.orchestra.run.smithi087.stdout:(29/177): librados-devel-debuginfo-19.0.0-763.g 1.0 MB/s | 108 kB 00:00 2024-01-20T08:23:59.497 INFO:teuthology.orchestra.run.smithi116.stdout:(26/177): libcephfs2-debuginfo-19.0.0-763.ga00c 5.5 MB/s | 5.3 MB 00:00 2024-01-20T08:23:59.589 INFO:teuthology.orchestra.run.smithi116.stdout:(27/177): libcephsqlite-19.0.0-763.ga00c9940.el 1.7 MB/s | 163 kB 00:00 2024-01-20T08:23:59.689 INFO:teuthology.orchestra.run.smithi116.stdout:(28/177): librados-devel-19.0.0-763.ga00c9940.e 1.2 MB/s | 127 kB 00:00 2024-01-20T08:23:59.772 INFO:teuthology.orchestra.run.smithi116.stdout:(29/177): librados-devel-debuginfo-19.0.0-763.g 1.3 MB/s | 108 kB 00:00 2024-01-20T08:24:00.716 INFO:teuthology.orchestra.run.smithi087.stdout:(30/177): libcephfs2-debuginfo-19.0.0-763.ga00c 2.4 MB/s | 5.3 MB 00:02 2024-01-20T08:24:00.991 INFO:teuthology.orchestra.run.smithi087.stdout:(31/177): libradosstriper1-19.0.0-763.ga00c9940 1.8 MB/s | 499 kB 00:00 2024-01-20T08:24:07.172 INFO:teuthology.orchestra.run.smithi116.stdout:(30/177): ceph-common-debuginfo-19.0.0-763.ga00 3.9 MB/s | 411 MB 01:44 2024-01-20T08:24:07.330 INFO:teuthology.orchestra.run.smithi116.stdout:(31/177): libradosstriper1-19.0.0-763.ga00c9940 3.1 MB/s | 499 kB 00:00 2024-01-20T08:24:07.641 INFO:teuthology.orchestra.run.smithi087.stdout:(32/177): librados2-debuginfo-19.0.0-763.ga00c9 5.1 MB/s | 44 MB 00:08 2024-01-20T08:24:08.699 INFO:teuthology.orchestra.run.smithi087.stdout:(33/177): librgw2-19.0.0-763.ga00c9940.el9.x86_ 4.5 MB/s | 4.8 MB 00:01 2024-01-20T08:24:08.799 INFO:teuthology.orchestra.run.smithi087.stdout:(34/177): python3-ceph-argparse-19.0.0-763.ga00 452 kB/s | 45 kB 00:00 2024-01-20T08:24:08.892 INFO:teuthology.orchestra.run.smithi087.stdout:(35/177): python3-ceph-common-19.0.0-763.ga00c9 1.4 MB/s | 130 kB 00:00 2024-01-20T08:24:08.983 INFO:teuthology.orchestra.run.smithi087.stdout:(36/177): python3-cephfs-19.0.0-763.ga00c9940.e 1.7 MB/s | 162 kB 00:00 2024-01-20T08:24:09.159 INFO:teuthology.orchestra.run.smithi087.stdout:(37/177): python3-cephfs-debuginfo-19.0.0-763.g 3.4 MB/s | 615 kB 00:00 2024-01-20T08:24:09.259 INFO:teuthology.orchestra.run.smithi087.stdout:(38/177): python3-rados-19.0.0-763.ga00c9940.el 3.1 MB/s | 320 kB 00:00 2024-01-20T08:24:09.443 INFO:teuthology.orchestra.run.smithi087.stdout:(39/177): python3-rados-debuginfo-19.0.0-763.ga 4.1 MB/s | 770 kB 00:00 2024-01-20T08:24:09.543 INFO:teuthology.orchestra.run.smithi087.stdout:(40/177): python3-rbd-19.0.0-763.ga00c9940.el9. 2.9 MB/s | 297 kB 00:00 2024-01-20T08:24:09.810 INFO:teuthology.orchestra.run.smithi087.stdout:(41/177): python3-rbd-debuginfo-19.0.0-763.ga00 4.2 MB/s | 1.1 MB 00:00 2024-01-20T08:24:09.902 INFO:teuthology.orchestra.run.smithi087.stdout:(42/177): python3-rgw-19.0.0-763.ga00c9940.el9. 1.1 MB/s | 99 kB 00:00 2024-01-20T08:24:10.010 INFO:teuthology.orchestra.run.smithi087.stdout:(43/177): python3-rgw-debuginfo-19.0.0-763.ga00 3.0 MB/s | 337 kB 00:00 2024-01-20T08:24:10.094 INFO:teuthology.orchestra.run.smithi087.stdout:(44/177): rbd-fuse-19.0.0-763.ga00c9940.el9.x86 1.0 MB/s | 85 kB 00:00 2024-01-20T08:24:10.219 INFO:teuthology.orchestra.run.smithi087.stdout:(45/177): rbd-fuse-debuginfo-19.0.0-763.ga00c99 4.0 MB/s | 509 kB 00:00 2024-01-20T08:24:10.514 INFO:teuthology.orchestra.run.smithi116.stdout:(32/177): librados2-debuginfo-19.0.0-763.ga00c9 4.1 MB/s | 44 MB 00:10 2024-01-20T08:24:10.678 INFO:teuthology.orchestra.run.smithi087.stdout:(46/177): rbd-mirror-19.0.0-763.ga00c9940.el9.x 6.7 MB/s | 3.1 MB 00:00 2024-01-20T08:24:11.755 INFO:teuthology.orchestra.run.smithi116.stdout:(33/177): librgw2-19.0.0-763.ga00c9940.el9.x86_ 3.9 MB/s | 4.8 MB 00:01 2024-01-20T08:24:11.822 INFO:teuthology.orchestra.run.smithi116.stdout:(34/177): python3-ceph-argparse-19.0.0-763.ga00 679 kB/s | 45 kB 00:00 2024-01-20T08:24:11.906 INFO:teuthology.orchestra.run.smithi116.stdout:(35/177): python3-ceph-common-19.0.0-763.ga00c9 1.5 MB/s | 130 kB 00:00 2024-01-20T08:24:11.989 INFO:teuthology.orchestra.run.smithi116.stdout:(36/177): python3-cephfs-19.0.0-763.ga00c9940.e 1.9 MB/s | 162 kB 00:00 2024-01-20T08:24:12.206 INFO:teuthology.orchestra.run.smithi116.stdout:(37/177): python3-cephfs-debuginfo-19.0.0-763.g 2.8 MB/s | 615 kB 00:00 2024-01-20T08:24:12.357 INFO:teuthology.orchestra.run.smithi116.stdout:(38/177): python3-rados-19.0.0-763.ga00c9940.el 2.1 MB/s | 320 kB 00:00 2024-01-20T08:24:12.582 INFO:teuthology.orchestra.run.smithi116.stdout:(39/177): python3-rados-debuginfo-19.0.0-763.ga 3.3 MB/s | 770 kB 00:00 2024-01-20T08:24:12.740 INFO:teuthology.orchestra.run.smithi116.stdout:(40/177): python3-rbd-19.0.0-763.ga00c9940.el9. 1.8 MB/s | 297 kB 00:00 2024-01-20T08:24:13.049 INFO:teuthology.orchestra.run.smithi116.stdout:(41/177): python3-rbd-debuginfo-19.0.0-763.ga00 3.6 MB/s | 1.1 MB 00:00 2024-01-20T08:24:13.132 INFO:teuthology.orchestra.run.smithi116.stdout:(42/177): python3-rgw-19.0.0-763.ga00c9940.el9. 1.2 MB/s | 99 kB 00:00 2024-01-20T08:24:13.283 INFO:teuthology.orchestra.run.smithi116.stdout:(43/177): python3-rgw-debuginfo-19.0.0-763.ga00 2.2 MB/s | 337 kB 00:00 2024-01-20T08:24:13.375 INFO:teuthology.orchestra.run.smithi116.stdout:(44/177): rbd-fuse-19.0.0-763.ga00c9940.el9.x86 930 kB/s | 85 kB 00:00 2024-01-20T08:24:13.550 INFO:teuthology.orchestra.run.smithi116.stdout:(45/177): rbd-fuse-debuginfo-19.0.0-763.ga00c99 2.8 MB/s | 509 kB 00:00 2024-01-20T08:24:14.400 INFO:teuthology.orchestra.run.smithi116.stdout:(46/177): rbd-mirror-19.0.0-763.ga00c9940.el9.x 3.6 MB/s | 3.1 MB 00:00 2024-01-20T08:24:15.528 INFO:teuthology.orchestra.run.smithi087.stdout:(47/177): librbd1-debuginfo-19.0.0-763.ga00c994 3.6 MB/s | 52 MB 00:14 2024-01-20T08:24:15.711 INFO:teuthology.orchestra.run.smithi087.stdout:(48/177): rbd-nbd-19.0.0-763.ga00c9940.el9.x86_ 932 kB/s | 170 kB 00:00 2024-01-20T08:24:16.028 INFO:teuthology.orchestra.run.smithi087.stdout:(49/177): rbd-nbd-debuginfo-19.0.0-763.ga00c994 3.9 MB/s | 1.2 MB 00:00 2024-01-20T08:24:16.104 INFO:teuthology.orchestra.run.smithi087.stdout:(50/177): ceph-grafana-dashboards-19.0.0-763.ga 379 kB/s | 28 kB 00:00 2024-01-20T08:24:16.179 INFO:teuthology.orchestra.run.smithi087.stdout:(51/177): ceph-mgr-cephadm-19.0.0-763.ga00c9940 1.7 MB/s | 133 kB 00:00 2024-01-20T08:24:16.234 INFO:teuthology.orchestra.run.smithi116.stdout:(47/177): librbd1-debuginfo-19.0.0-763.ga00c994 5.8 MB/s | 52 MB 00:08 2024-01-20T08:24:16.326 INFO:teuthology.orchestra.run.smithi116.stdout:(48/177): rbd-nbd-19.0.0-763.ga00c9940.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-01-20T08:24:16.642 INFO:teuthology.orchestra.run.smithi116.stdout:(49/177): rbd-nbd-debuginfo-19.0.0-763.ga00c994 4.0 MB/s | 1.2 MB 00:00 2024-01-20T08:24:16.670 INFO:teuthology.orchestra.run.smithi087.stdout:(52/177): ceph-mgr-dashboard-19.0.0-763.ga00c99 3.5 MB/s | 1.7 MB 00:00 2024-01-20T08:24:16.709 INFO:teuthology.orchestra.run.smithi116.stdout:(50/177): ceph-grafana-dashboards-19.0.0-763.ga 428 kB/s | 28 kB 00:00 2024-01-20T08:24:16.793 INFO:teuthology.orchestra.run.smithi116.stdout:(51/177): ceph-mgr-cephadm-19.0.0-763.ga00c9940 1.6 MB/s | 133 kB 00:00 2024-01-20T08:24:17.252 INFO:teuthology.orchestra.run.smithi116.stdout:(52/177): ceph-mgr-dashboard-19.0.0-763.ga00c99 3.8 MB/s | 1.7 MB 00:00 2024-01-20T08:24:18.596 INFO:teuthology.orchestra.run.smithi087.stdout:(53/177): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-01-20T08:24:18.721 INFO:teuthology.orchestra.run.smithi087.stdout:(54/177): ceph-mgr-modules-core-19.0.0-763.ga00 1.9 MB/s | 245 kB 00:00 2024-01-20T08:24:18.831 INFO:teuthology.orchestra.run.smithi087.stdout:(55/177): ceph-mgr-rook-19.0.0-763.ga00c9940.el 460 kB/s | 51 kB 00:00 2024-01-20T08:24:18.940 INFO:teuthology.orchestra.run.smithi087.stdout:(56/177): ceph-prometheus-alerts-19.0.0-763.ga0 139 kB/s | 15 kB 00:00 2024-01-20T08:24:19.057 INFO:teuthology.orchestra.run.smithi087.stdout:(57/177): ceph-volume-19.0.0-763.ga00c9940.el9. 2.2 MB/s | 262 kB 00:00 2024-01-20T08:24:19.168 INFO:teuthology.orchestra.run.smithi116.stdout:(53/177): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-01-20T08:24:19.257 INFO:teuthology.orchestra.run.smithi087.stdout:(58/177): cephadm-19.0.0-763.ga00c9940.el9.noar 3.6 MB/s | 742 kB 00:00 2024-01-20T08:24:19.285 INFO:teuthology.orchestra.run.smithi116.stdout:(54/177): ceph-mgr-modules-core-19.0.0-763.ga00 2.0 MB/s | 245 kB 00:00 2024-01-20T08:24:19.358 INFO:teuthology.orchestra.run.smithi087.stdout:(59/177): cephfs-top-19.0.0-763.ga00c9940.el9.n 182 kB/s | 18 kB 00:00 2024-01-20T08:24:19.369 INFO:teuthology.orchestra.run.smithi116.stdout:(55/177): ceph-mgr-rook-19.0.0-763.ga00c9940.el 610 kB/s | 51 kB 00:00 2024-01-20T08:24:19.436 INFO:teuthology.orchestra.run.smithi116.stdout:(56/177): ceph-prometheus-alerts-19.0.0-763.ga0 227 kB/s | 15 kB 00:00 2024-01-20T08:24:19.511 INFO:teuthology.orchestra.run.smithi116.stdout:(57/177): ceph-volume-19.0.0-763.ga00c9940.el9. 3.4 MB/s | 262 kB 00:00 2024-01-20T08:24:19.686 INFO:teuthology.orchestra.run.smithi116.stdout:(58/177): cephadm-19.0.0-763.ga00c9940.el9.noar 4.1 MB/s | 742 kB 00:00 2024-01-20T08:24:19.745 INFO:teuthology.orchestra.run.smithi116.stdout:(59/177): cephfs-top-19.0.0-763.ga00c9940.el9.n 312 kB/s | 18 kB 00:00 2024-01-20T08:24:19.792 INFO:teuthology.orchestra.run.smithi087.stdout:(60/177): rbd-mirror-debuginfo-19.0.0-763.ga00c 5.7 MB/s | 52 MB 00:09 2024-01-20T08:24:19.887 INFO:teuthology.orchestra.run.smithi116.stdout:(60/177): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-01-20T08:24:19.946 INFO:teuthology.orchestra.run.smithi116.stdout:(61/177): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-01-20T08:24:19.950 INFO:teuthology.orchestra.run.smithi087.stdout:(61/177): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 662 kB 00:00 2024-01-20T08:24:19.979 INFO:teuthology.orchestra.run.smithi116.stdout:(62/177): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-01-20T08:24:19.983 INFO:teuthology.orchestra.run.smithi087.stdout:(62/177): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-01-20T08:24:20.004 INFO:teuthology.orchestra.run.smithi116.stdout:(63/177): elfutils-libelf-devel-0.190-2.el9.x86 815 kB/s | 20 kB 00:00 2024-01-20T08:24:20.009 INFO:teuthology.orchestra.run.smithi087.stdout:(63/177): elfutils-libelf-devel-0.190-2.el9.x86 820 kB/s | 20 kB 00:00 2024-01-20T08:24:20.046 INFO:teuthology.orchestra.run.smithi116.stdout:(64/177): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-01-20T08:24:20.050 INFO:teuthology.orchestra.run.smithi087.stdout:(64/177): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-01-20T08:24:20.072 INFO:teuthology.orchestra.run.smithi116.stdout:(65/177): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-01-20T08:24:20.076 INFO:teuthology.orchestra.run.smithi087.stdout:(65/177): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 32 kB 00:00 2024-01-20T08:24:20.110 INFO:teuthology.orchestra.run.smithi087.stdout:(66/177): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2024-01-20T08:24:20.180 INFO:teuthology.orchestra.run.smithi116.stdout:(66/177): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-01-20T08:24:20.206 INFO:teuthology.orchestra.run.smithi116.stdout:(67/177): flexiblas-openblas-openmp-3.0.4-8.el9 683 kB/s | 17 kB 00:00 2024-01-20T08:24:20.226 INFO:teuthology.orchestra.run.smithi087.stdout:(67/177): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-01-20T08:24:20.239 INFO:teuthology.orchestra.run.smithi116.stdout:(68/177): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-01-20T08:24:20.252 INFO:teuthology.orchestra.run.smithi087.stdout:(68/177): flexiblas-openblas-openmp-3.0.4-8.el9 122 kB/s | 17 kB 00:00 2024-01-20T08:24:20.273 INFO:teuthology.orchestra.run.smithi116.stdout:(69/177): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-01-20T08:24:20.277 INFO:teuthology.orchestra.run.smithi087.stdout:(69/177): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-01-20T08:24:20.302 INFO:teuthology.orchestra.run.smithi087.stdout:(70/177): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-01-20T08:24:20.306 INFO:teuthology.orchestra.run.smithi116.stdout:(70/177): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-01-20T08:24:20.327 INFO:teuthology.orchestra.run.smithi087.stdout:(71/177): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-01-20T08:24:20.353 INFO:teuthology.orchestra.run.smithi087.stdout:(72/177): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-01-20T08:24:20.356 INFO:teuthology.orchestra.run.smithi116.stdout:(71/177): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-01-20T08:24:20.378 INFO:teuthology.orchestra.run.smithi087.stdout:(73/177): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-01-20T08:24:20.399 INFO:teuthology.orchestra.run.smithi116.stdout:(72/177): libtool-2.4.6-45.el9.x86_64.rpm 13 MB/s | 578 kB 00:00 2024-01-20T08:24:20.404 INFO:teuthology.orchestra.run.smithi087.stdout:(74/177): libuuid-devel-2.37.4-15.el9.x86_64.rp 496 kB/s | 24 kB 00:00 2024-01-20T08:24:20.429 INFO:teuthology.orchestra.run.smithi087.stdout:(75/177): libzstd-devel-1.5.1-2.el9.x86_64.rpm 924 kB/s | 46 kB 00:00 2024-01-20T08:24:20.432 INFO:teuthology.orchestra.run.smithi116.stdout:(73/177): libuuid-devel-2.37.4-15.el9.x86_64.rp 740 kB/s | 24 kB 00:00 2024-01-20T08:24:20.454 INFO:teuthology.orchestra.run.smithi087.stdout:(76/177): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-01-20T08:24:20.466 INFO:teuthology.orchestra.run.smithi116.stdout:(74/177): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-01-20T08:24:20.479 INFO:teuthology.orchestra.run.smithi087.stdout:(77/177): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-01-20T08:24:20.505 INFO:teuthology.orchestra.run.smithi087.stdout:(78/177): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2024-01-20T08:24:20.508 INFO:teuthology.orchestra.run.smithi116.stdout:(75/177): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-01-20T08:24:20.550 INFO:teuthology.orchestra.run.smithi116.stdout:(76/177): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-01-20T08:24:20.600 INFO:teuthology.orchestra.run.smithi116.stdout:(77/177): openblas-0.3.21-2.el9.x86_64.rpm 683 kB/s | 34 kB 00:00 2024-01-20T08:24:20.605 INFO:teuthology.orchestra.run.smithi087.stdout:(79/177): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-01-20T08:24:20.672 INFO:teuthology.orchestra.run.smithi087.stdout:(80/177): openssl-devel-3.0.7-25.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-01-20T08:24:20.706 INFO:teuthology.orchestra.run.smithi087.stdout:(81/177): postgresql-13.11-1.el9.x86_64.rpm 16 MB/s | 1.5 MB 00:00 2024-01-20T08:24:20.739 INFO:teuthology.orchestra.run.smithi087.stdout:(82/177): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-01-20T08:24:20.742 INFO:teuthology.orchestra.run.smithi116.stdout:(78/177): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-01-20T08:24:20.792 INFO:teuthology.orchestra.run.smithi087.stdout:(83/177): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-01-20T08:24:20.867 INFO:teuthology.orchestra.run.smithi116.stdout:(79/177): openssl-devel-3.0.7-25.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-01-20T08:24:20.926 INFO:teuthology.orchestra.run.smithi116.stdout:(80/177): postgresql-13.11-1.el9.x86_64.rpm 26 MB/s | 1.5 MB 00:00 2024-01-20T08:24:20.968 INFO:teuthology.orchestra.run.smithi116.stdout:(81/177): postgresql-contrib-13.11-1.el9.x86_64 19 MB/s | 811 kB 00:00 2024-01-20T08:24:21.001 INFO:teuthology.orchestra.run.smithi116.stdout:(82/177): postgresql-private-libs-13.11-1.el9.x 3.9 MB/s | 134 kB 00:00 2024-01-20T08:24:21.065 INFO:teuthology.orchestra.run.smithi087.stdout:(84/177): postgresql-server-13.11-1.el9.x86_64. 17 MB/s | 5.7 MB 00:00 2024-01-20T08:24:21.136 INFO:teuthology.orchestra.run.smithi116.stdout:(83/177): postgresql-server-13.11-1.el9.x86_64. 42 MB/s | 5.7 MB 00:00 2024-01-20T08:24:21.169 INFO:teuthology.orchestra.run.smithi116.stdout:(84/177): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-01-20T08:24:21.345 INFO:teuthology.orchestra.run.smithi116.stdout:(85/177): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-01-20T08:24:21.357 INFO:teuthology.orchestra.run.smithi087.stdout:(85/177): python3-attrs-20.3.0-7.el9.noarch.rpm 138 kB/s | 78 kB 00:00 2024-01-20T08:24:21.378 INFO:teuthology.orchestra.run.smithi116.stdout:(86/177): python3-jinja2-2.11.3-4.el9.noarch.rp 6.8 MB/s | 228 kB 00:00 2024-01-20T08:24:21.412 INFO:teuthology.orchestra.run.smithi116.stdout:(87/177): python3-jsonpatch-1.21-16.el9.noarch. 732 kB/s | 24 kB 00:00 2024-01-20T08:24:21.445 INFO:teuthology.orchestra.run.smithi116.stdout:(88/177): python3-jsonpointer-2.0-4.el9.noarch. 545 kB/s | 18 kB 00:00 2024-01-20T08:24:21.479 INFO:teuthology.orchestra.run.smithi116.stdout:(89/177): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-01-20T08:24:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:(90/177): python3-markupsafe-1.1.1-12.el9.x86_6 946 kB/s | 32 kB 00:00 2024-01-20T08:24:21.616 INFO:teuthology.orchestra.run.smithi087.stdout:(86/177): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2024-01-20T08:24:21.638 INFO:teuthology.orchestra.run.smithi116.stdout:(91/177): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-01-20T08:24:21.671 INFO:teuthology.orchestra.run.smithi116.stdout:(92/177): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.4 MB/s | 217 kB 00:00 2024-01-20T08:24:21.705 INFO:teuthology.orchestra.run.smithi116.stdout:(93/177): python3-oauthlib-3.1.1-5.el9.noarch.r 5.1 MB/s | 174 kB 00:00 2024-01-20T08:24:21.738 INFO:teuthology.orchestra.run.smithi116.stdout:(94/177): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-01-20T08:24:21.772 INFO:teuthology.orchestra.run.smithi116.stdout:(95/177): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-01-20T08:24:21.814 INFO:teuthology.orchestra.run.smithi116.stdout:(96/177): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-01-20T08:24:21.840 INFO:teuthology.orchestra.run.smithi116.stdout:(97/177): python3-pytz-2021.1-5.el9.noarch.rpm 1.9 MB/s | 47 kB 00:00 2024-01-20T08:24:21.873 INFO:teuthology.orchestra.run.smithi116.stdout:(98/177): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-01-20T08:24:21.891 INFO:teuthology.orchestra.run.smithi087.stdout:(87/177): python3-jinja2-2.11.3-4.el9.noarch.rp 431 kB/s | 228 kB 00:00 2024-01-20T08:24:21.933 INFO:teuthology.orchestra.run.smithi087.stdout:(88/177): python3-jsonpatch-1.21-16.el9.noarch. 77 kB/s | 24 kB 00:00 2024-01-20T08:24:21.958 INFO:teuthology.orchestra.run.smithi087.stdout:(89/177): python3-jsonpointer-2.0-4.el9.noarch. 270 kB/s | 18 kB 00:00 2024-01-20T08:24:21.992 INFO:teuthology.orchestra.run.smithi087.stdout:(90/177): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-01-20T08:24:22.017 INFO:teuthology.orchestra.run.smithi087.stdout:(91/177): python3-markupsafe-1.1.1-12.el9.x86_6 540 kB/s | 32 kB 00:00 2024-01-20T08:24:22.159 INFO:teuthology.orchestra.run.smithi087.stdout:(92/177): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-01-20T08:24:22.184 INFO:teuthology.orchestra.run.smithi087.stdout:(93/177): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-01-20T08:24:22.210 INFO:teuthology.orchestra.run.smithi087.stdout:(94/177): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-01-20T08:24:22.215 INFO:teuthology.orchestra.run.smithi116.stdout:(99/177): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-01-20T08:24:22.226 INFO:teuthology.orchestra.run.smithi087.stdout:(95/177): python3-prettytable-0.7.2-27.el9.noar 990 kB/s | 41 kB 00:00 2024-01-20T08:24:22.240 INFO:teuthology.orchestra.run.smithi116.stdout:(100/177): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-01-20T08:24:22.252 INFO:teuthology.orchestra.run.smithi087.stdout:(96/177): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-01-20T08:24:22.266 INFO:teuthology.orchestra.run.smithi116.stdout:(101/177): python3-wcwidth-0.2.5-8.el9.noarch.r 1.6 MB/s | 41 kB 00:00 2024-01-20T08:24:22.277 INFO:teuthology.orchestra.run.smithi087.stdout:(97/177): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-01-20T08:24:22.294 INFO:teuthology.orchestra.run.smithi087.stdout:(98/177): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-01-20T08:24:22.307 INFO:teuthology.orchestra.run.smithi116.stdout:(102/177): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-01-20T08:24:22.311 INFO:teuthology.orchestra.run.smithi087.stdout:(99/177): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-01-20T08:24:22.341 INFO:teuthology.orchestra.run.smithi116.stdout:(103/177): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-01-20T08:24:22.383 INFO:teuthology.orchestra.run.smithi116.stdout:(104/177): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.0 MB/s | 87 kB 00:00 2024-01-20T08:24:22.420 INFO:teuthology.orchestra.run.smithi087.stdout:(100/177): python3-toml-0.10.2-6.el9.noarch.rpm 359 kB/s | 39 kB 00:00 2024-01-20T08:24:22.425 INFO:teuthology.orchestra.run.smithi116.stdout:(105/177): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-01-20T08:24:22.459 INFO:teuthology.orchestra.run.smithi116.stdout:(106/177): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-01-20T08:24:22.501 INFO:teuthology.orchestra.run.smithi116.stdout:(107/177): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-01-20T08:24:22.534 INFO:teuthology.orchestra.run.smithi116.stdout:(108/177): jq-1.6-16.el9.x86_64.rpm 5.4 MB/s | 186 kB 00:00 2024-01-20T08:24:22.593 INFO:teuthology.orchestra.run.smithi116.stdout:(109/177): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-01-20T08:24:22.627 INFO:teuthology.orchestra.run.smithi116.stdout:(110/177): libquadmath-11.4.1-3.el9.x86_64.rpm 5.7 MB/s | 194 kB 00:00 2024-01-20T08:24:22.637 INFO:teuthology.orchestra.run.smithi087.stdout:(101/177): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-01-20T08:24:22.662 INFO:teuthology.orchestra.run.smithi087.stdout:(102/177): python3-wcwidth-0.2.5-8.el9.noarch.r 171 kB/s | 41 kB 00:00 2024-01-20T08:24:22.671 INFO:teuthology.orchestra.run.smithi116.stdout:(111/177): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.8 MB/s | 217 kB 00:00 2024-01-20T08:24:22.687 INFO:teuthology.orchestra.run.smithi087.stdout:(103/177): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-01-20T08:24:22.713 INFO:teuthology.orchestra.run.smithi087.stdout:(104/177): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-01-20T08:24:22.713 INFO:teuthology.orchestra.run.smithi116.stdout:(112/177): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-01-20T08:24:22.738 INFO:teuthology.orchestra.run.smithi087.stdout:(105/177): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.7 MB/s | 87 kB 00:00 2024-01-20T08:24:22.763 INFO:teuthology.orchestra.run.smithi087.stdout:(106/177): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-01-20T08:24:22.771 INFO:teuthology.orchestra.run.smithi116.stdout:(113/177): python3-cryptography-36.0.1-4.el9.x8 20 MB/s | 1.1 MB 00:00 2024-01-20T08:24:22.788 INFO:teuthology.orchestra.run.smithi087.stdout:(107/177): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-01-20T08:24:22.813 INFO:teuthology.orchestra.run.smithi087.stdout:(108/177): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-01-20T08:24:22.822 INFO:teuthology.orchestra.run.smithi116.stdout:(114/177): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-01-20T08:24:22.839 INFO:teuthology.orchestra.run.smithi087.stdout:(109/177): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-01-20T08:24:22.855 INFO:teuthology.orchestra.run.smithi116.stdout:(115/177): python3-pycparser-2.20-6.el9.noarch. 3.6 MB/s | 124 kB 00:00 2024-01-20T08:24:22.864 INFO:teuthology.orchestra.run.smithi087.stdout:(110/177): libgfortran-11.4.1-3.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-01-20T08:24:22.880 INFO:teuthology.orchestra.run.smithi116.stdout:(116/177): python3-requests-2.25.1-7.el9.noarch 4.4 MB/s | 113 kB 00:00 2024-01-20T08:24:22.889 INFO:teuthology.orchestra.run.smithi087.stdout:(111/177): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 194 kB 00:00 2024-01-20T08:24:22.915 INFO:teuthology.orchestra.run.smithi087.stdout:(112/177): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-01-20T08:24:22.922 INFO:teuthology.orchestra.run.smithi116.stdout:(117/177): python3-urllib3-1.26.5-5.el9.noarch. 4.4 MB/s | 187 kB 00:00 2024-01-20T08:24:22.940 INFO:teuthology.orchestra.run.smithi087.stdout:(113/177): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-01-20T08:24:22.974 INFO:teuthology.orchestra.run.smithi087.stdout:(114/177): python3-cryptography-36.0.1-4.el9.x8 20 MB/s | 1.1 MB 00:00 2024-01-20T08:24:22.999 INFO:teuthology.orchestra.run.smithi087.stdout:(115/177): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-01-20T08:24:23.024 INFO:teuthology.orchestra.run.smithi087.stdout:(116/177): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-01-20T08:24:23.040 INFO:teuthology.orchestra.run.smithi116.stdout:(118/177): python3-devel-3.9.17-2.el9.x86_64.rp 2.1 MB/s | 245 kB 00:00 2024-01-20T08:24:23.050 INFO:teuthology.orchestra.run.smithi087.stdout:(117/177): python3-requests-2.25.1-7.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-01-20T08:24:23.075 INFO:teuthology.orchestra.run.smithi087.stdout:(118/177): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-01-20T08:24:23.350 INFO:teuthology.orchestra.run.smithi087.stdout:(119/177): python3-devel-3.9.17-2.el9.x86_64.rp 818 kB/s | 245 kB 00:00 2024-01-20T08:24:23.376 INFO:teuthology.orchestra.run.smithi087.stdout:(120/177): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-01-20T08:24:23.482 INFO:teuthology.orchestra.run.smithi116.stdout:(119/177): rbd-mirror-debuginfo-19.0.0-763.ga00 5.7 MB/s | 52 MB 00:09 2024-01-20T08:24:23.534 INFO:teuthology.orchestra.run.smithi087.stdout:(121/177): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-01-20T08:24:23.557 INFO:teuthology.orchestra.run.smithi116.stdout:(120/177): lua-devel-5.4.4-4.el9.x86_64.rpm 43 kB/s | 22 kB 00:00 2024-01-20T08:24:23.576 INFO:teuthology.orchestra.run.smithi087.stdout:(122/177): libarrow-doc-9.0.0-10.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-01-20T08:24:23.627 INFO:teuthology.orchestra.run.smithi087.stdout:(123/177): liboath-2.6.7-2.el9.x86_64.rpm 975 kB/s | 49 kB 00:00 2024-01-20T08:24:23.668 INFO:teuthology.orchestra.run.smithi087.stdout:(124/177): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-01-20T08:24:23.736 INFO:teuthology.orchestra.run.smithi087.stdout:(125/177): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-01-20T08:24:23.763 INFO:teuthology.orchestra.run.smithi087.stdout:(126/177): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-01-20T08:24:23.803 INFO:teuthology.orchestra.run.smithi087.stdout:(127/177): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-01-20T08:24:23.844 INFO:teuthology.orchestra.run.smithi087.stdout:(128/177): python3-asyncssh-2.13.2-3.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-01-20T08:24:23.870 INFO:teuthology.orchestra.run.smithi087.stdout:(129/177): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-01-20T08:24:23.895 INFO:teuthology.orchestra.run.smithi087.stdout:(130/177): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-01-20T08:24:23.920 INFO:teuthology.orchestra.run.smithi087.stdout:(131/177): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-01-20T08:24:23.946 INFO:teuthology.orchestra.run.smithi087.stdout:(132/177): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-01-20T08:24:23.971 INFO:teuthology.orchestra.run.smithi087.stdout:(133/177): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-01-20T08:24:24.005 INFO:teuthology.orchestra.run.smithi087.stdout:(134/177): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-01-20T08:24:24.030 INFO:teuthology.orchestra.run.smithi087.stdout:(135/177): python3-colorama-0.4.4-12.el9.noarch 600 kB/s | 35 kB 00:00 2024-01-20T08:24:24.055 INFO:teuthology.orchestra.run.smithi087.stdout:(136/177): python3-google-auth-2.26.0-1.el9.noa 3.9 MB/s | 202 kB 00:00 2024-01-20T08:24:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:(137/177): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-01-20T08:24:24.090 INFO:teuthology.orchestra.run.smithi116.stdout:(121/177): gperftools-libs-2.9.1-2.el9.x86_64.r 508 kB/s | 309 kB 00:00 2024-01-20T08:24:24.097 INFO:teuthology.orchestra.run.smithi087.stdout:(138/177): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-01-20T08:24:24.122 INFO:teuthology.orchestra.run.smithi087.stdout:(139/177): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-01-20T08:24:24.148 INFO:teuthology.orchestra.run.smithi087.stdout:(140/177): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-01-20T08:24:24.165 INFO:teuthology.orchestra.run.smithi087.stdout:(141/177): python3-jaraco-text-3.2.0-6.el9.noar 465 kB/s | 20 kB 00:00 2024-01-20T08:24:24.199 INFO:teuthology.orchestra.run.smithi087.stdout:(142/177): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-01-20T08:24:24.232 INFO:teuthology.orchestra.run.smithi087.stdout:(143/177): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-01-20T08:24:24.240 INFO:teuthology.orchestra.run.smithi116.stdout:(122/177): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-01-20T08:24:24.258 INFO:teuthology.orchestra.run.smithi087.stdout:(144/177): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-01-20T08:24:24.274 INFO:teuthology.orchestra.run.smithi087.stdout:(145/177): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-01-20T08:24:24.300 INFO:teuthology.orchestra.run.smithi087.stdout:(146/177): python3-pecan-1.4.2-3.el9.noarch.rpm 6.3 MB/s | 272 kB 00:00 2024-01-20T08:24:24.325 INFO:teuthology.orchestra.run.smithi087.stdout:(147/177): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-01-20T08:24:24.350 INFO:teuthology.orchestra.run.smithi087.stdout:(148/177): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-01-20T08:24:24.376 INFO:teuthology.orchestra.run.smithi087.stdout:(149/177): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-01-20T08:24:24.383 INFO:teuthology.orchestra.run.smithi116.stdout:(123/177): liboath-2.6.7-2.el9.x86_64.rpm 343 kB/s | 49 kB 00:00 2024-01-20T08:24:24.401 INFO:teuthology.orchestra.run.smithi087.stdout:(150/177): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-01-20T08:24:24.427 INFO:teuthology.orchestra.run.smithi087.stdout:(151/177): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-01-20T08:24:24.451 INFO:teuthology.orchestra.run.smithi087.stdout:(152/177): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-01-20T08:24:24.477 INFO:teuthology.orchestra.run.smithi087.stdout:(153/177): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-01-20T08:24:24.502 INFO:teuthology.orchestra.run.smithi087.stdout:(154/177): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-01-20T08:24:24.536 INFO:teuthology.orchestra.run.smithi087.stdout:(155/177): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-01-20T08:24:24.553 INFO:teuthology.orchestra.run.smithi087.stdout:(156/177): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-01-20T08:24:24.575 INFO:teuthology.orchestra.run.smithi116.stdout:(124/177): libunwind-1.6.2-1.el9.x86_64.rpm 351 kB/s | 67 kB 00:00 2024-01-20T08:24:24.586 INFO:teuthology.orchestra.run.smithi087.stdout:(157/177): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-01-20T08:24:24.612 INFO:teuthology.orchestra.run.smithi087.stdout:(158/177): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-01-20T08:24:24.637 INFO:teuthology.orchestra.run.smithi087.stdout:(159/177): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-01-20T08:24:24.704 INFO:teuthology.orchestra.run.smithi087.stdout:(160/177): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-01-20T08:24:25.183 INFO:teuthology.orchestra.run.smithi116.stdout:(125/177): luarocks-3.9.2-1.el9.noarch.rpm 248 kB/s | 151 kB 00:00 2024-01-20T08:24:25.421 INFO:teuthology.orchestra.run.smithi087.stdout:(161/177): librbd1-19.0.0-763.ga00c9940.el9.x86 4.4 MB/s | 3.1 MB 00:00 2024-01-20T08:24:25.488 INFO:teuthology.orchestra.run.smithi087.stdout:(162/177): librados2-19.0.0-763.ga00c9940.el9.x 3.9 MB/s | 3.4 MB 00:00 2024-01-20T08:24:25.513 INFO:teuthology.orchestra.run.smithi087.stdout:(163/177): libacl-devel-2.3.1-4.el9.x86_64.rpm 737 kB/s | 68 kB 00:00 2024-01-20T08:24:25.563 INFO:teuthology.orchestra.run.smithi087.stdout:(164/177): NetworkManager-1.45.10-1.el9.x86_64. 30 MB/s | 2.2 MB 00:00 2024-01-20T08:24:25.607 INFO:teuthology.orchestra.run.smithi087.stdout:(165/177): NetworkManager-libnm-1.45.10-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-01-20T08:24:25.630 INFO:teuthology.orchestra.run.smithi087.stdout:(166/177): NetworkManager-team-1.45.10-1.el9.x8 650 kB/s | 42 kB 00:00 2024-01-20T08:24:25.642 INFO:teuthology.orchestra.run.smithi116.stdout:(126/177): libarrow-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-01-20T08:24:25.656 INFO:teuthology.orchestra.run.smithi087.stdout:(167/177): NetworkManager-tui-1.45.10-1.el9.x86 4.8 MB/s | 246 kB 00:00 2024-01-20T08:24:25.681 INFO:teuthology.orchestra.run.smithi087.stdout:(168/177): acl-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 69 kB 00:00 2024-01-20T08:24:25.707 INFO:teuthology.orchestra.run.smithi087.stdout:(169/177): elfutils-debuginfod-client-0.190-2.e 700 kB/s | 35 kB 00:00 2024-01-20T08:24:25.732 INFO:teuthology.orchestra.run.smithi087.stdout:(170/177): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-01-20T08:24:25.757 INFO:teuthology.orchestra.run.smithi087.stdout:(171/177): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-01-20T08:24:25.782 INFO:teuthology.orchestra.run.smithi087.stdout:(172/177): libacl-2.3.1-4.el9.x86_64.rpm 439 kB/s | 22 kB 00:00 2024-01-20T08:24:25.816 INFO:teuthology.orchestra.run.smithi087.stdout:(173/177): openssl-3.0.7-25.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-01-20T08:24:25.858 INFO:teuthology.orchestra.run.smithi087.stdout:(174/177): openssl-libs-3.0.7-25.el9.x86_64.rpm 29 MB/s | 2.1 MB 00:00 2024-01-20T08:24:25.883 INFO:teuthology.orchestra.run.smithi087.stdout:(175/177): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2024-01-20T08:24:25.934 INFO:teuthology.orchestra.run.smithi116.stdout:(127/177): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 838 kB 00:00 2024-01-20T08:24:25.959 INFO:teuthology.orchestra.run.smithi087.stdout:(176/177): xfsprogs-6.3.0-1.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-01-20T08:24:26.034 INFO:teuthology.orchestra.run.smithi116.stdout:(128/177): python3-asyncssh-2.13.2-3.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-01-20T08:24:26.151 INFO:teuthology.orchestra.run.smithi116.stdout:(129/177): python3-bcrypt-3.2.2-1.el9.x86_64.rp 200 kB/s | 43 kB 00:00 2024-01-20T08:24:26.201 INFO:teuthology.orchestra.run.smithi116.stdout:(130/177): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-01-20T08:24:26.293 INFO:teuthology.orchestra.run.smithi116.stdout:(131/177): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-01-20T08:24:26.452 INFO:teuthology.orchestra.run.smithi116.stdout:(132/177): python3-cheroot-8.6.0-4.el9.noarch.r 686 kB/s | 172 kB 00:00 2024-01-20T08:24:26.519 INFO:teuthology.orchestra.run.smithi116.stdout:(133/177): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-01-20T08:24:26.678 INFO:teuthology.orchestra.run.smithi116.stdout:(134/177): python3-colorama-0.4.4-12.el9.noarch 223 kB/s | 35 kB 00:00 2024-01-20T08:24:26.703 INFO:teuthology.orchestra.run.smithi116.stdout:(135/177): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.2 MB/s | 303 kB 00:00 2024-01-20T08:24:26.845 INFO:teuthology.orchestra.run.smithi116.stdout:(136/177): python3-google-auth-2.26.0-1.el9.noa 1.2 MB/s | 202 kB 00:00 2024-01-20T08:24:26.870 INFO:teuthology.orchestra.run.smithi116.stdout:(137/177): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-01-20T08:24:26.979 INFO:teuthology.orchestra.run.smithi116.stdout:(138/177): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-01-20T08:24:26.995 INFO:teuthology.orchestra.run.smithi116.stdout:(139/177): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-01-20T08:24:27.096 INFO:teuthology.orchestra.run.smithi116.stdout:(140/177): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-01-20T08:24:27.146 INFO:teuthology.orchestra.run.smithi116.stdout:(141/177): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2024-01-20T08:24:27.288 INFO:teuthology.orchestra.run.smithi116.stdout:(142/177): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2024-01-20T08:24:27.438 INFO:teuthology.orchestra.run.smithi116.stdout:(143/177): python3-more-itertools-8.12.0-2.el9. 525 kB/s | 79 kB 00:00 2024-01-20T08:24:27.489 INFO:teuthology.orchestra.run.smithi116.stdout:(144/177): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-01-20T08:24:27.531 INFO:teuthology.orchestra.run.smithi116.stdout:(145/177): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-01-20T08:24:27.656 INFO:teuthology.orchestra.run.smithi116.stdout:(146/177): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-01-20T08:24:27.682 INFO:teuthology.orchestra.run.smithi116.stdout:(147/177): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-01-20T08:24:27.832 INFO:teuthology.orchestra.run.smithi116.stdout:(148/177): python3-pyOpenSSL-21.0.0-1.el9.noarc 514 kB/s | 90 kB 00:00 2024-01-20T08:24:27.857 INFO:teuthology.orchestra.run.smithi116.stdout:(149/177): python3-pyperclip-1.8.2-2.el9.noarch 161 kB/s | 28 kB 00:00 2024-01-20T08:24:27.999 INFO:teuthology.orchestra.run.smithi116.stdout:(150/177): python3-repoze-lru-0.7-16.el9.noarch 184 kB/s | 31 kB 00:00 2024-01-20T08:24:28.032 INFO:teuthology.orchestra.run.smithi116.stdout:(151/177): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-01-20T08:24:28.133 INFO:teuthology.orchestra.run.smithi116.stdout:(152/177): python3-rsa-4.9-2.el9.noarch.rpm 442 kB/s | 59 kB 00:00 2024-01-20T08:24:28.158 INFO:teuthology.orchestra.run.smithi116.stdout:(153/177): python3-tempora-5.0.0-2.el9.noarch.r 285 kB/s | 36 kB 00:00 2024-01-20T08:24:28.233 INFO:teuthology.orchestra.run.smithi116.stdout:(154/177): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-01-20T08:24:28.342 INFO:teuthology.orchestra.run.smithi116.stdout:(155/177): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-01-20T08:24:28.367 INFO:teuthology.orchestra.run.smithi116.stdout:(156/177): python3-websocket-client-1.2.3-2.el9 672 kB/s | 90 kB 00:00 2024-01-20T08:24:28.509 INFO:teuthology.orchestra.run.smithi116.stdout:(157/177): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-01-20T08:24:28.618 INFO:teuthology.orchestra.run.smithi116.stdout:(158/177): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2024-01-20T08:24:28.695 INFO:teuthology.orchestra.run.smithi116.stdout:(159/177): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-01-20T08:24:29.169 INFO:teuthology.orchestra.run.smithi116.stdout:(160/177): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-01-20T08:24:29.394 INFO:teuthology.orchestra.run.smithi116.stdout:(161/177): librados2-19.0.0-763.ga00c9940.el9.x 4.8 MB/s | 3.4 MB 00:00 2024-01-20T08:24:29.427 INFO:teuthology.orchestra.run.smithi116.stdout:(162/177): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-01-20T08:24:29.502 INFO:teuthology.orchestra.run.smithi116.stdout:(163/177): NetworkManager-1.45.10-1.el9.x86_64. 30 MB/s | 2.2 MB 00:00 2024-01-20T08:24:29.570 INFO:teuthology.orchestra.run.smithi116.stdout:(164/177): NetworkManager-libnm-1.45.10-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-01-20T08:24:29.603 INFO:teuthology.orchestra.run.smithi116.stdout:(165/177): NetworkManager-team-1.45.10-1.el9.x8 1.2 MB/s | 42 kB 00:00 2024-01-20T08:24:29.654 INFO:teuthology.orchestra.run.smithi116.stdout:(166/177): NetworkManager-tui-1.45.10-1.el9.x86 4.8 MB/s | 246 kB 00:00 2024-01-20T08:24:29.687 INFO:teuthology.orchestra.run.smithi116.stdout:(167/177): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-01-20T08:24:29.721 INFO:teuthology.orchestra.run.smithi116.stdout:(168/177): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-01-20T08:24:29.755 INFO:teuthology.orchestra.run.smithi116.stdout:(169/177): elfutils-libelf-0.190-2.el9.x86_64.r 5.6 MB/s | 191 kB 00:00 2024-01-20T08:24:29.806 INFO:teuthology.orchestra.run.smithi116.stdout:(170/177): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-01-20T08:24:29.839 INFO:teuthology.orchestra.run.smithi116.stdout:(171/177): libacl-2.3.1-4.el9.x86_64.rpm 661 kB/s | 22 kB 00:00 2024-01-20T08:24:29.897 INFO:teuthology.orchestra.run.smithi116.stdout:(172/177): openssl-3.0.7-25.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-01-20T08:24:29.981 INFO:teuthology.orchestra.run.smithi116.stdout:(173/177): openssl-libs-3.0.7-25.el9.x86_64.rpm 26 MB/s | 2.1 MB 00:00 2024-01-20T08:24:30.023 INFO:teuthology.orchestra.run.smithi116.stdout:(174/177): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-01-20T08:24:30.081 INFO:teuthology.orchestra.run.smithi116.stdout:(175/177): librbd1-19.0.0-763.ga00c9940.el9.x86 3.4 MB/s | 3.1 MB 00:00 2024-01-20T08:24:30.115 INFO:teuthology.orchestra.run.smithi116.stdout:(176/177): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-01-20T08:24:53.773 INFO:teuthology.orchestra.run.smithi087.stdout:(177/177): ceph-test-debuginfo-19.0.0-763.ga00c 10 MB/s | 664 MB 01:04 2024-01-20T08:24:53.777 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-01-20T08:24:53.778 INFO:teuthology.orchestra.run.smithi087.stdout:Total 14 MB/s | 2.1 GB 02:34 2024-01-20T08:24:53.810 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-01-20T08:24:54.095 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-01-20T08:24:54.095 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-01-20T08:24:59.620 INFO:teuthology.orchestra.run.smithi116.stdout:(177/177): ceph-test-debuginfo-19.0.0-763.ga00c 10 MB/s | 664 MB 01:06 2024-01-20T08:24:59.623 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-01-20T08:24:59.623 INFO:teuthology.orchestra.run.smithi116.stdout:Total 14 MB/s | 2.1 GB 02:38 2024-01-20T08:24:59.653 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-01-20T08:24:59.926 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-01-20T08:24:59.926 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-01-20T08:25:07.473 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-01-20T08:25:07.474 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-01-20T08:25:13.276 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-01-20T08:25:13.276 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-01-20T08:25:27.983 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-01-20T08:25:28.108 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 1/193 2024-01-20T08:25:28.174 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 2/193 2024-01-20T08:25:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/193 2024-01-20T08:25:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/193 2024-01-20T08:25:28.308 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/193 2024-01-20T08:25:28.333 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/193 2024-01-20T08:25:28.387 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/193 2024-01-20T08:25:28.753 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/193 2024-01-20T08:25:28.785 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:19.0.0-763.ga00c9940.el9.x86_64 9/193 2024-01-20T08:25:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:19.0.0-763.ga00c9940.el9.x86_64 9/193 2024-01-20T08:25:28.952 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 10/193 2024-01-20T08:25:29.043 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 10/193 2024-01-20T08:25:29.085 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:19.0.0-763.ga00c9940.el9.x86_64 11/193 2024-01-20T08:25:29.169 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/193 2024-01-20T08:25:29.254 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/193 2024-01-20T08:25:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/193 2024-01-20T08:25:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/193 2024-01-20T08:25:29.534 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 16/193 2024-01-20T08:25:29.637 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 16/193 2024-01-20T08:25:29.660 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_ 17/193 2024-01-20T08:25:29.730 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_ 17/193 2024-01-20T08:25:29.931 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/193 2024-01-20T08:25:30.089 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-20T08:25:30.132 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-20T08:25:30.256 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-20T08:25:30.301 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-20T08:25:30.586 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-20T08:25:30.622 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/193 2024-01-20T08:25:30.683 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/193 2024-01-20T08:25:30.736 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/193 2024-01-20T08:25:30.767 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 24/193 2024-01-20T08:25:30.810 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/193 2024-01-20T08:25:30.850 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/193 2024-01-20T08:25:30.923 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 27/193 2024-01-20T08:25:30.958 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 28/193 2024-01-20T08:25:30.999 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 29/193 2024-01-20T08:25:31.028 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/193 2024-01-20T08:25:31.098 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/193 2024-01-20T08:25:31.133 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:19.0.0-763.ga00c9940.el9.x 32/193 2024-01-20T08:25:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:19.0.0-763.ga00c9940.el9 33/193 2024-01-20T08:25:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:19.0.0-763.ga00c9940.el9.x86_64 34/193 2024-01-20T08:25:31.283 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/193 2024-01-20T08:25:31.368 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/193 2024-01-20T08:25:31.511 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/193 2024-01-20T08:25:31.564 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/193 2024-01-20T08:25:32.097 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/193 2024-01-20T08:25:32.176 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/193 2024-01-20T08:25:32.207 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 41/193 2024-01-20T08:25:32.247 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/193 2024-01-20T08:25:32.285 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:19.0.0-763.ga00c9940.el9.x86_64 43/193 2024-01-20T08:25:32.328 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/193 2024-01-20T08:25:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/193 2024-01-20T08:25:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 46/193 2024-01-20T08:25:32.616 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : re2-1:20211101-3.el9.x86_64 47/193 2024-01-20T08:25:32.655 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 48/193 2024-01-20T08:25:32.710 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/193 2024-01-20T08:25:32.783 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/193 2024-01-20T08:25:32.814 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/193 2024-01-20T08:25:32.851 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/193 2024-01-20T08:25:32.898 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/193 2024-01-20T08:25:32.934 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/193 2024-01-20T08:25:32.977 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/193 2024-01-20T08:25:33.019 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/193 2024-01-20T08:25:33.155 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/193 2024-01-20T08:25:33.200 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/193 2024-01-20T08:25:33.230 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/193 2024-01-20T08:25:33.271 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/193 2024-01-20T08:25:33.367 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/193 2024-01-20T08:25:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 62/193 2024-01-20T08:25:33.446 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/193 2024-01-20T08:25:33.512 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/193 2024-01-20T08:25:33.677 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 65/193 2024-01-20T08:25:33.723 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-01-20T08:25:33.848 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 66/193 2024-01-20T08:25:33.858 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 1/193 2024-01-20T08:25:33.906 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/193 2024-01-20T08:25:33.929 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 2/193 2024-01-20T08:25:33.961 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/193 2024-01-20T08:25:33.967 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/193 2024-01-20T08:25:34.029 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/193 2024-01-20T08:25:34.062 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/193 2024-01-20T08:25:34.088 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/193 2024-01-20T08:25:34.142 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/193 2024-01-20T08:25:34.157 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/193 2024-01-20T08:25:34.201 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 70/193 2024-01-20T08:25:34.332 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/193 2024-01-20T08:25:34.375 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 72/193 2024-01-20T08:25:34.410 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 73/193 2024-01-20T08:25:34.445 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 74/193 2024-01-20T08:25:34.478 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/193 2024-01-20T08:25:34.481 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : jq-1.6-16.el9.x86_64 75/193 2024-01-20T08:25:34.511 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.9-15.el9.x86_64 76/193 2024-01-20T08:25:34.523 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librados2-2:19.0.0-763.ga00c9940.el9.x86_64 9/193 2024-01-20T08:25:34.539 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 77/193 2024-01-20T08:25:34.582 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 78/193 2024-01-20T08:25:34.622 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : uuid-1.6.2-55.el9.x86_64 79/193 2024-01-20T08:25:34.644 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:19.0.0-763.ga00c9940.el9.x86_64 9/193 2024-01-20T08:25:34.653 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 80/193 2024-01-20T08:25:34.673 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 10/193 2024-01-20T08:25:34.683 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/193 2024-01-20T08:25:34.756 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 10/193 2024-01-20T08:25:34.761 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/193 2024-01-20T08:25:34.792 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rados-2:19.0.0-763.ga00c9940.el9.x86_64 11/193 2024-01-20T08:25:34.810 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/193 2024-01-20T08:25:34.843 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/193 2024-01-20T08:25:34.884 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/193 2024-01-20T08:25:34.885 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/193 2024-01-20T08:25:34.919 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/193 2024-01-20T08:25:34.969 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/193 2024-01-20T08:25:35.009 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/193 2024-01-20T08:25:35.047 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/193 2024-01-20T08:25:35.143 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/193 2024-01-20T08:25:35.150 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/193 2024-01-20T08:25:35.186 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/193 2024-01-20T08:25:35.206 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/193 2024-01-20T08:25:35.214 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 16/193 2024-01-20T08:25:35.300 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 16/193 2024-01-20T08:25:35.323 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_ 17/193 2024-01-20T08:25:35.385 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_ 17/193 2024-01-20T08:25:35.546 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/193 2024-01-20T08:25:35.661 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-20T08:25:35.729 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-20T08:25:35.948 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/193 2024-01-20T08:25:35.985 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-20T08:25:35.995 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/193 2024-01-20T08:25:36.032 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/193 2024-01-20T08:25:36.081 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/193 2024-01-20T08:25:36.159 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-20T08:25:36.169 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 95/193 2024-01-20T08:25:36.208 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 96/193 2024-01-20T08:25:36.384 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.21-2.el9.x86_64 97/193 2024-01-20T08:25:36.414 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 98/193 2024-01-20T08:25:36.433 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-20T08:25:36.460 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/193 2024-01-20T08:25:36.492 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 99/193 2024-01-20T08:25:36.522 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/193 2024-01-20T08:25:36.591 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/193 2024-01-20T08:25:36.648 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 24/193 2024-01-20T08:25:36.691 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/193 2024-01-20T08:25:36.731 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/193 2024-01-20T08:25:36.804 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 27/193 2024-01-20T08:25:36.838 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 28/193 2024-01-20T08:25:36.875 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 29/193 2024-01-20T08:25:36.899 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/193 2024-01-20T08:25:36.961 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/193 2024-01-20T08:25:36.996 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-common-2:19.0.0-763.ga00c9940.el9.x 32/193 2024-01-20T08:25:37.031 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-argparse-2:19.0.0-763.ga00c9940.el9 33/193 2024-01-20T08:25:37.059 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cephfs-2:19.0.0-763.ga00c9940.el9.x86_64 34/193 2024-01-20T08:25:37.129 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/193 2024-01-20T08:25:37.164 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 100/193 2024-01-20T08:25:37.206 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/193 2024-01-20T08:25:37.288 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 101/193 2024-01-20T08:25:37.351 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/193 2024-01-20T08:25:37.401 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/193 2024-01-20T08:25:37.929 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/193 2024-01-20T08:25:38.013 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/193 2024-01-20T08:25:38.045 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 41/193 2024-01-20T08:25:38.085 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/193 2024-01-20T08:25:38.123 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados-devel-2:19.0.0-763.ga00c9940.el9.x86_64 43/193 2024-01-20T08:25:38.165 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/193 2024-01-20T08:25:38.293 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/193 2024-01-20T08:25:38.332 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 46/193 2024-01-20T08:25:38.437 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : re2-1:20211101-3.el9.x86_64 47/193 2024-01-20T08:25:38.476 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 48/193 2024-01-20T08:25:38.532 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/193 2024-01-20T08:25:38.550 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 102/193 2024-01-20T08:25:38.605 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/193 2024-01-20T08:25:38.625 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 103/193 2024-01-20T08:25:38.635 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/193 2024-01-20T08:25:38.661 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/193 2024-01-20T08:25:38.672 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/193 2024-01-20T08:25:38.694 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 105/193 2024-01-20T08:25:38.719 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/193 2024-01-20T08:25:38.755 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/193 2024-01-20T08:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/193 2024-01-20T08:25:38.841 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/193 2024-01-20T08:25:38.975 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/193 2024-01-20T08:25:39.000 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/193 2024-01-20T08:25:39.020 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/193 2024-01-20T08:25:39.040 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 107/193 2024-01-20T08:25:39.051 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/193 2024-01-20T08:25:39.092 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/193 2024-01-20T08:25:39.131 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 107/193 2024-01-20T08:25:39.161 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:19.0.0-763.ga00c9940.el9.x86_64 108/193 2024-01-20T08:25:39.188 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/193 2024-01-20T08:25:39.197 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/193 2024-01-20T08:25:39.226 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 62/193 2024-01-20T08:25:39.269 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/193 2024-01-20T08:25:39.300 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/193 2024-01-20T08:25:39.395 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 65/193 2024-01-20T08:25:39.444 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 66/193 2024-01-20T08:25:39.501 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/193 2024-01-20T08:25:39.556 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/193 2024-01-20T08:25:39.633 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/193 2024-01-20T08:25:39.660 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 111/193 2024-01-20T08:25:39.714 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 111/193 2024-01-20T08:25:39.736 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:19.0.0-763.ga00c9940.el9.x86_64 112/193 2024-01-20T08:25:39.758 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/193 2024-01-20T08:25:39.806 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 70/193 2024-01-20T08:25:39.931 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/193 2024-01-20T08:25:39.968 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 72/193 2024-01-20T08:25:40.001 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 73/193 2024-01-20T08:25:40.039 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 74/193 2024-01-20T08:25:40.068 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : jq-1.6-16.el9.x86_64 75/193 2024-01-20T08:25:40.099 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fuse-2.9.9-15.el9.x86_64 76/193 2024-01-20T08:25:40.126 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 77/193 2024-01-20T08:25:40.165 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 78/193 2024-01-20T08:25:40.200 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : uuid-1.6.2-55.el9.x86_64 79/193 2024-01-20T08:25:40.231 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 80/193 2024-01-20T08:25:40.261 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/193 2024-01-20T08:25:40.339 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/193 2024-01-20T08:25:40.372 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/193 2024-01-20T08:25:40.405 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/193 2024-01-20T08:25:40.446 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/193 2024-01-20T08:25:40.481 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/193 2024-01-20T08:25:40.570 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/193 2024-01-20T08:25:40.609 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/193 2024-01-20T08:25:40.711 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/193 2024-01-20T08:25:40.760 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/193 2024-01-20T08:25:41.491 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/193 2024-01-20T08:25:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/193 2024-01-20T08:25:41.575 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/193 2024-01-20T08:25:41.623 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/193 2024-01-20T08:25:41.703 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 95/193 2024-01-20T08:25:41.743 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 96/193 2024-01-20T08:25:41.826 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 113/193 2024-01-20T08:25:41.866 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 113/193 2024-01-20T08:25:41.916 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-0.3.21-2.el9.x86_64 97/193 2024-01-20T08:25:41.940 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 98/193 2024-01-20T08:25:42.018 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 99/193 2024-01-20T08:25:42.393 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 113/193 2024-01-20T08:25:42.427 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 114/193 2024-01-20T08:25:42.538 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 114/193 2024-01-20T08:25:42.538 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-20T08:25:42.538 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-20T08:25:42.538 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:42.578 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 115/193 2024-01-20T08:25:42.665 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 100/193 2024-01-20T08:25:42.780 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 101/193 2024-01-20T08:25:44.029 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 102/193 2024-01-20T08:25:44.112 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 103/193 2024-01-20T08:25:44.140 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/193 2024-01-20T08:25:44.173 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 105/193 2024-01-20T08:25:44.480 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/193 2024-01-20T08:25:44.511 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 107/193 2024-01-20T08:25:44.584 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 107/193 2024-01-20T08:25:44.616 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rbd-2:19.0.0-763.ga00c9940.el9.x86_64 108/193 2024-01-20T08:25:44.652 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/193 2024-01-20T08:25:45.070 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/193 2024-01-20T08:25:45.106 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 111/193 2024-01-20T08:25:45.176 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 111/193 2024-01-20T08:25:45.199 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rgw-2:19.0.0-763.ga00c9940.el9.x86_64 112/193 2024-01-20T08:25:47.316 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 113/193 2024-01-20T08:25:47.346 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 113/193 2024-01-20T08:25:47.885 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 113/193 2024-01-20T08:25:47.924 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 114/193 2024-01-20T08:25:48.020 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 114/193 2024-01-20T08:25:48.020 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-20T08:25:48.020 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-20T08:25:48.021 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:25:48.058 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 115/193 2024-01-20T08:25:54.515 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 115/193 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /root 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2024-01-20T08:25:54.516 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:54.672 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 116/193 2024-01-20T08:25:54.687 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 116/193 2024-01-20T08:25:54.687 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:25:54.687 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-20T08:25:54.687 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:56.167 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 117/193 2024-01-20T08:25:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 117/193 2024-01-20T08:25:56.192 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:25:56.192 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-20T08:25:56.192 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-20T08:25:56.192 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-20T08:25:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:56.401 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 118/193 2024-01-20T08:25:56.423 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 118/193 2024-01-20T08:25:56.424 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:25:56.424 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-20T08:25:56.424 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-20T08:25:56.424 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-20T08:25:56.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:56.868 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 119/193 2024-01-20T08:25:56.890 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 119/193 2024-01-20T08:25:56.890 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:25:56.890 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-20T08:25:56.890 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-20T08:25:56.890 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-20T08:25:56.890 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:56.995 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : automake-1.16.2-8.el9.noarch 120/193 2024-01-20T08:25:57.359 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:19.0.0-763.ga00c9940.el9.noarch 121/193 2024-01-20T08:25:57.382 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:19.0.0-763.ga00c9940.el9.noarch 121/193 2024-01-20T08:25:57.412 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-763.ga00c9940.el 122/193 2024-01-20T08:25:57.472 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-763.ga00c9940.e 123/193 2024-01-20T08:25:57.513 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noar 124/193 2024-01-20T08:25:57.734 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noar 124/193 2024-01-20T08:25:57.764 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.no 125/193 2024-01-20T08:25:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.no 125/193 2024-01-20T08:25:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c 126/193 2024-01-20T08:25:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c 126/193 2024-01-20T08:25:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-763.ga00c9940.el9 127/193 2024-01-20T08:25:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 128/193 2024-01-20T08:25:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 128/193 2024-01-20T08:25:59.056 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:25:59.056 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-20T08:25:59.056 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-20T08:25:59.057 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-20T08:25:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:25:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 129/193 2024-01-20T08:25:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 129/193 2024-01-20T08:25:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:19.0.0-763.ga00c9940.el9.x86_64 130/193 2024-01-20T08:25:59.859 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 115/193 2024-01-20T08:25:59.859 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /sys 2024-01-20T08:25:59.859 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /proc 2024-01-20T08:25:59.860 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /mnt 2024-01-20T08:25:59.860 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /var/tmp 2024-01-20T08:25:59.860 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /home 2024-01-20T08:25:59.860 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /root 2024-01-20T08:25:59.860 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /tmp 2024-01-20T08:25:59.860 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:25:59.951 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/193 2024-01-20T08:25:59.977 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 132/193 2024-01-20T08:26:00.000 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 132/193 2024-01-20T08:26:00.000 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:00.000 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-20T08:26:00.000 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-20T08:26:00.000 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-20T08:26:00.000 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:26:00.020 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 116/193 2024-01-20T08:26:00.035 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 116/193 2024-01-20T08:26:00.035 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:00.035 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-20T08:26:00.035 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:00.044 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-763.ga00c99 133/193 2024-01-20T08:26:00.066 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-763.ga00c99 133/193 2024-01-20T08:26:00.066 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:00.066 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-20T08:26:00.066 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:26:00.112 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 134/193 2024-01-20T08:26:00.134 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 134/193 2024-01-20T08:26:00.134 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:00.134 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-20T08:26:00.134 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-20T08:26:00.134 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-20T08:26:00.134 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:26:00.446 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 135/193 2024-01-20T08:26:00.468 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 135/193 2024-01-20T08:26:00.468 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:00.468 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-20T08:26:00.468 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-20T08:26:00.468 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-20T08:26:00.468 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:26:01.516 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 117/193 2024-01-20T08:26:01.541 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 117/193 2024-01-20T08:26:01.541 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:01.541 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-20T08:26:01.541 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-20T08:26:01.541 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-20T08:26:01.541 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:01.749 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 118/193 2024-01-20T08:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 118/193 2024-01-20T08:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-20T08:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-20T08:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-20T08:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:02.209 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 119/193 2024-01-20T08:26:02.231 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 119/193 2024-01-20T08:26:02.231 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:02.231 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-20T08:26:02.231 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-20T08:26:02.231 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-20T08:26:02.231 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:02.319 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : automake-1.16.2-8.el9.noarch 120/193 2024-01-20T08:26:02.676 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephadm-2:19.0.0-763.ga00c9940.el9.noarch 121/193 2024-01-20T08:26:02.709 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephadm-2:19.0.0-763.ga00c9940.el9.noarch 121/193 2024-01-20T08:26:02.738 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-763.ga00c9940.el 122/193 2024-01-20T08:26:02.800 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-763.ga00c9940.e 123/193 2024-01-20T08:26:02.823 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noar 124/193 2024-01-20T08:26:03.045 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noar 124/193 2024-01-20T08:26:03.074 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.no 125/193 2024-01-20T08:26:04.037 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.no 125/193 2024-01-20T08:26:04.066 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c 126/193 2024-01-20T08:26:04.165 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c 126/193 2024-01-20T08:26:04.320 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-763.ga00c9940.el9 127/193 2024-01-20T08:26:04.343 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 128/193 2024-01-20T08:26:04.365 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 128/193 2024-01-20T08:26:04.365 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:04.365 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-20T08:26:04.365 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-20T08:26:04.366 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-20T08:26:04.366 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:04.410 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 129/193 2024-01-20T08:26:04.420 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 129/193 2024-01-20T08:26:04.471 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-2:19.0.0-763.ga00c9940.el9.x86_64 130/193 2024-01-20T08:26:04.808 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:19.0.0-763.ga00c9940.el9.x86_64 136/193 2024-01-20T08:26:04.849 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 137/193 2024-01-20T08:26:04.889 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:19.0.0-763.ga00c9940.el9.x86_64 138/193 2024-01-20T08:26:04.922 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/193 2024-01-20T08:26:04.949 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/193 2024-01-20T08:26:05.123 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-shell-2:19.0.0-763.ga00c9940.el9.x86_64 141/193 2024-01-20T08:26:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/193 2024-01-20T08:26:05.262 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/193 2024-01-20T08:26:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 143/193 2024-01-20T08:26:05.287 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 132/193 2024-01-20T08:26:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:19.0.0-763.ga00c9940.el9.x86_6 144/193 2024-01-20T08:26:05.309 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 132/193 2024-01-20T08:26:05.310 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:05.310 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-20T08:26:05.310 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-20T08:26:05.310 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-20T08:26:05.310 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:05.327 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 145/193 2024-01-20T08:26:05.346 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-763.ga00c99 133/193 2024-01-20T08:26:05.357 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 146/193 2024-01-20T08:26:05.368 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-763.ga00c99 133/193 2024-01-20T08:26:05.368 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:05.368 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-20T08:26:05.368 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:05.414 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 134/193 2024-01-20T08:26:05.434 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 147/193 2024-01-20T08:26:05.435 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 134/193 2024-01-20T08:26:05.435 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:05.436 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-20T08:26:05.436 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-20T08:26:05.436 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-20T08:26:05.436 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:05.489 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/193 2024-01-20T08:26:05.522 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/193 2024-01-20T08:26:05.739 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 135/193 2024-01-20T08:26:05.761 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 135/193 2024-01-20T08:26:05.761 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-20T08:26:05.761 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-20T08:26:05.761 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-20T08:26:05.762 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-20T08:26:05.762 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-20T08:26:06.179 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-20T08:26:06.214 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-20T08:26:06.268 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 151/193 2024-01-20T08:26:07.075 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-top-2:19.0.0-763.ga00c9940.el9.noarch 152/193 2024-01-20T08:26:10.059 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-test-2:19.0.0-763.ga00c9940.el9.x86_64 136/193 2024-01-20T08:26:10.100 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 137/193 2024-01-20T08:26:10.141 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-nbd-2:19.0.0-763.ga00c9940.el9.x86_64 138/193 2024-01-20T08:26:10.174 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/193 2024-01-20T08:26:10.201 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/193 2024-01-20T08:26:10.385 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-shell-2:19.0.0-763.ga00c9940.el9.x86_64 141/193 2024-01-20T08:26:10.508 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/193 2024-01-20T08:26:10.539 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 143/193 2024-01-20T08:26:10.564 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs-devel-2:19.0.0-763.ga00c9940.el9.x86_6 144/193 2024-01-20T08:26:10.604 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 145/193 2024-01-20T08:26:10.659 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 146/193 2024-01-20T08:26:10.747 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 147/193 2024-01-20T08:26:10.821 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/193 2024-01-20T08:26:10.866 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/193 2024-01-20T08:26:11.435 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-20T08:26:11.481 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-20T08:26:11.519 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-20T08:26:11.612 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 151/193 2024-01-20T08:26:12.420 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-top-2:19.0.0-763.ga00c9940.el9.noarch 152/193 2024-01-20T08:26:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 153/193 2024-01-20T08:26:19.404 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 153/193 2024-01-20T08:26:53.084 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-debuginfo-2:19.0.0-763.ga00c9940.el9.x 154/193 2024-01-20T08:26:53.738 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-debuginfo-2:19.0.0-763.ga00c9940.el9 155/193 2024-01-20T08:26:53.915 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x 156/193 2024-01-20T08:26:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-7 157/193 2024-01-20T08:26:58.541 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-base-debuginfo-2:19.0.0-763.ga00c9940.el9.x 154/193 2024-01-20T08:26:59.315 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-common-debuginfo-2:19.0.0-763.ga00c9940.el9 155/193 2024-01-20T08:26:59.705 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x 156/193 2024-01-20T08:27:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 158/193 2024-01-20T08:27:04.380 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-7 157/193 2024-01-20T08:27:06.475 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 158/193 2024-01-20T08:27:10.170 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 159/193 2024-01-20T08:27:16.352 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 159/193 2024-01-20T08:27:37.844 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 160/193 2024-01-20T08:27:44.059 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 160/193 2024-01-20T08:27:54.234 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 161/193 2024-01-20T08:28:00.416 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 161/193 2024-01-20T08:29:00.316 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-763.ga00c9940.el 162/193 2024-01-20T08:29:00.925 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-debuginfo-2:19.0.0-763.ga00c9940.el9.x 163/193 2024-01-20T08:29:00.973 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-763.ga00c9940.el9. 164/193 2024-01-20T08:29:05.649 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-debuginfo-2:19.0.0-763.ga00c9940. 165/193 2024-01-20T08:29:06.346 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-763.ga00c9940.el 162/193 2024-01-20T08:29:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-test-debuginfo-2:19.0.0-763.ga00c9940.el9.x 163/193 2024-01-20T08:29:07.508 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-763.ga00c9940.el9. 164/193 2024-01-20T08:29:11.463 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados2-debuginfo-2:19.0.0-763.ga00c9940.el9.x 166/193 2024-01-20T08:29:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librbd1-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 167/193 2024-01-20T08:29:12.152 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-763.ga00c9940. 168/193 2024-01-20T08:29:12.435 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados-devel-debuginfo-2:19.0.0-763.ga00c9940. 165/193 2024-01-20T08:29:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-debuginfo-2:19.0.0-763.ga00c9940.e 169/193 2024-01-20T08:29:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-763.ga00c9940.el9 170/193 2024-01-20T08:29:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-763.ga00c9940.el9 171/193 2024-01-20T08:29:18.056 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados2-debuginfo-2:19.0.0-763.ga00c9940.el9.x 166/193 2024-01-20T08:29:18.144 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librbd1-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 167/193 2024-01-20T08:29:18.242 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-763.ga00c9940. 168/193 2024-01-20T08:29:18.385 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rados-debuginfo-2:19.0.0-763.ga00c9940.e 169/193 2024-01-20T08:29:18.444 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-763.ga00c9940.el9 170/193 2024-01-20T08:29:18.507 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 172/193 2024-01-20T08:29:18.523 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-763.ga00c9940.el9 171/193 2024-01-20T08:29:18.856 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-763.ga00c9940.el9. 173/193 2024-01-20T08:29:19.084 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 174/193 2024-01-20T08:29:19.813 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 175/193 2024-01-20T08:29:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flex-2.6.4-9.el9.x86_64 176/193 2024-01-20T08:29:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : bison-3.7.4-5.el9.x86_64 177/193 2024-01-20T08:29:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 178/193 2024-01-20T08:29:19.996 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-20T08:29:20.023 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 180/193 2024-01-20T08:29:20.023 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-20T08:29:20.107 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-20T08:29:20.135 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 182/193 2024-01-20T08:29:20.183 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 183/193 2024-01-20T08:29:20.216 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/193 2024-01-20T08:29:20.258 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 185/193 2024-01-20T08:29:20.258 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-20T08:29:20.342 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-20T08:29:20.367 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 187/193 2024-01-20T08:29:20.374 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-20T08:29:20.374 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-20T08:29:20.561 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-20T08:29:20.561 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-20T08:29:20.645 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-20T08:29:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 190/193 2024-01-20T08:29:20.696 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 191/193 2024-01-20T08:29:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 192/193 2024-01-20T08:29:20.795 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-20T08:29:24.066 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 172/193 2024-01-20T08:29:24.212 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-763.ga00c9940.el9. 173/193 2024-01-20T08:29:24.242 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 174/193 2024-01-20T08:29:24.295 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 175/193 2024-01-20T08:29:24.391 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flex-2.6.4-9.el9.x86_64 176/193 2024-01-20T08:29:24.429 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : bison-3.7.4-5.el9.x86_64 177/193 2024-01-20T08:29:24.454 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 178/193 2024-01-20T08:29:24.479 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-20T08:29:24.506 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 180/193 2024-01-20T08:29:24.506 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-20T08:29:24.599 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-20T08:29:24.626 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 182/193 2024-01-20T08:29:24.669 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 183/193 2024-01-20T08:29:24.708 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/193 2024-01-20T08:29:24.733 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 185/193 2024-01-20T08:29:24.733 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-20T08:29:24.817 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-20T08:29:24.842 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 187/193 2024-01-20T08:29:24.848 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-20T08:29:24.848 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-20T08:29:25.031 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-20T08:29:25.031 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-20T08:29:25.112 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-20T08:29:25.136 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 190/193 2024-01-20T08:29:25.171 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 191/193 2024-01-20T08:29:25.205 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 192/193 2024-01-20T08:29:25.270 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-20T08:29:29.771 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-20T08:29:29.771 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:19.0.0-763.ga00c9940.el9.x86_64 1/193 2024-01-20T08:29:29.771 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 2/193 2024-01-20T08:29:29.771 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-763.ga00c9940.el9.x 3/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 4/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-763.ga00c9940.el9 5/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 6/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 7/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x 8/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-763.ga00c99 9/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-7 10/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 11/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 12/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 13/193 2024-01-20T08:29:29.772 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 14/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 15/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 16/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 17/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 18/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 19/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-763.ga00c9940.el 20/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 21/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:19.0.0-763.ga00c9940.el9.x86_64 22/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-763.ga00c9940.el9.x 23/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 24/193 2024-01-20T08:29:29.773 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-shell-2:19.0.0-763.ga00c9940.el9.x86_64 25/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:19.0.0-763.ga00c9940.el9.x86_6 26/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 27/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-763.ga00c9940.el9. 28/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 29/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:19.0.0-763.ga00c9940.el9.x86_64 30/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-763.ga00c9940. 31/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-debuginfo-2:19.0.0-763.ga00c9940.el9.x 32/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_ 33/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 34/193 2024-01-20T08:29:29.774 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 35/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:19.0.0-763.ga00c9940.el9 36/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:19.0.0-763.ga00c9940.el9.x 37/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:19.0.0-763.ga00c9940.el9.x86_64 38/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-763.ga00c9940. 39/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:19.0.0-763.ga00c9940.el9.x86_64 40/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-763.ga00c9940.e 41/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:19.0.0-763.ga00c9940.el9.x86_64 42/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-763.ga00c9940.el9 43/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:19.0.0-763.ga00c9940.el9.x86_64 44/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-763.ga00c9940.el9 45/193 2024-01-20T08:29:29.775 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 46/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 47/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 48/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-763.ga00c9940.el9. 49/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:19.0.0-763.ga00c9940.el9.x86_64 50/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 51/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-763.ga00c9940.e 52/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noar 53/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.no 54/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c 55/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-763.ga00c9940.el9 56/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 57/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-763.ga00c9940.el 58/193 2024-01-20T08:29:29.776 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 59/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:19.0.0-763.ga00c9940.el9.noarch 60/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-top-2:19.0.0-763.ga00c9940.el9.noarch 61/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 64/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 65/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 67/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 68/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 69/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/193 2024-01-20T08:29:29.777 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 74/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 75/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 76/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 77/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-5.4.4-4.el9.x86_64 78/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 79/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 80/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 81/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-13.11-1.el9.x86_64 82/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 83/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 84/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 85/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 86/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 87/193 2024-01-20T08:29:29.778 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 88/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 89/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 90/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 91/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 92/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 93/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 95/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 96/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 97/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 98/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 99/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 100/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 101/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 102/193 2024-01-20T08:29:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 103/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 104/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 105/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 106/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 108/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 109/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : jq-1.6-16.el9.x86_64 110/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 111/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 112/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 113/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 114/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 115/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ply-3.11-14.el9.noarch 116/193 2024-01-20T08:29:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 117/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 118/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 119/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 120/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 121/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 122/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 123/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 124/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 125/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 127/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 128/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 129/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 130/193 2024-01-20T08:29:29.781 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 131/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 132/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 133/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 134/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 135/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 136/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 137/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 138/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 139/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 140/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 141/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 142/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 143/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 144/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 145/193 2024-01-20T08:29:29.782 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 146/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 147/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 148/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 149/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 150/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 151/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 152/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 153/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 154/193 2024-01-20T08:29:29.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 155/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 156/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 158/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : re2-1:20211101-3.el9.x86_64 160/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 161/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:19.0.0-763.ga00c9940.el9.x86_64 162/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 163/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 164/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 165/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 166/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 167/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 168/193 2024-01-20T08:29:29.784 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 169/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 170/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 171/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 172/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 174/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : acl-2.3.1-4.el9.x86_64 176/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : acl-2.3.1-3.el9.x86_64 177/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 178/193 2024-01-20T08:29:29.785 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 180/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 181/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 182/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 183/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 184/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 185/193 2024-01-20T08:29:29.786 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/193 2024-01-20T08:29:29.787 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/193 2024-01-20T08:29:29.787 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/193 2024-01-20T08:29:29.787 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/193 2024-01-20T08:29:29.787 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 190/193 2024-01-20T08:29:29.787 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 191/193 2024-01-20T08:29:29.787 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 192/193 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 193/193 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:32.187 INFO:teuthology.orchestra.run.smithi087.stdout: acl-2.3.1-4.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-01-20T08:29:32.188 INFO:teuthology.orchestra.run.smithi087.stdout: automake-1.16.2-8.el9.noarch 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: bison-3.7.4-5.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.189 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.191 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: flex-2.6.4-9.el9.x86_64 2024-01-20T08:29:32.192 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: jq-1.6-16.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-20T08:29:32.193 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-20T08:29:32.194 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: lua-5.4.4-4.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-13.11-1.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-20T08:29:32.195 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-20T08:29:32.196 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-20T08:29:32.197 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply-3.11-14.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-20T08:29:32.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-20T08:29:32.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: re2-1:20211101-3.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-20T08:29:32.200 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-20T08:29:32.201 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-20T08:29:32.201 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-20T08:29:32.201 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-01-20T08:29:32.843 DEBUG:teuthology.parallel:result is None 2024-01-20T08:29:34.082 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-20T08:29:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-2:19.0.0-763.ga00c9940.el9.x86_64 1/193 2024-01-20T08:29:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 2/193 2024-01-20T08:29:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-763.ga00c9940.el9.x 3/193 2024-01-20T08:29:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 4/193 2024-01-20T08:29:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-763.ga00c9940.el9 5/193 2024-01-20T08:29:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 6/193 2024-01-20T08:29:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 7/193 2024-01-20T08:29:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x 8/193 2024-01-20T08:29:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-763.ga00c99 9/193 2024-01-20T08:29:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-7 10/193 2024-01-20T08:29:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 11/193 2024-01-20T08:29:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 12/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 13/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 14/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 15/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 16/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 17/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 18/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 19/193 2024-01-20T08:29:34.085 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-763.ga00c9940.el 20/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 21/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-test-2:19.0.0-763.ga00c9940.el9.x86_64 22/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-763.ga00c9940.el9.x 23/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 24/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-shell-2:19.0.0-763.ga00c9940.el9.x86_64 25/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs-devel-2:19.0.0-763.ga00c9940.el9.x86_6 26/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 27/193 2024-01-20T08:29:34.086 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-763.ga00c9940.el9. 28/193 2024-01-20T08:29:34.087 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 29/193 2024-01-20T08:29:34.110 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados-devel-2:19.0.0-763.ga00c9940.el9.x86_64 30/193 2024-01-20T08:29:34.110 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-763.ga00c9940. 31/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-debuginfo-2:19.0.0-763.ga00c9940.el9.x 32/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_ 33/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 34/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 35/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-argparse-2:19.0.0-763.ga00c9940.el9 36/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-common-2:19.0.0-763.ga00c9940.el9.x 37/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cephfs-2:19.0.0-763.ga00c9940.el9.x86_64 38/193 2024-01-20T08:29:34.111 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-763.ga00c9940. 39/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rados-2:19.0.0-763.ga00c9940.el9.x86_64 40/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-763.ga00c9940.e 41/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rbd-2:19.0.0-763.ga00c9940.el9.x86_64 42/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-763.ga00c9940.el9 43/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rgw-2:19.0.0-763.ga00c9940.el9.x86_64 44/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-763.ga00c9940.el9 45/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 46/193 2024-01-20T08:29:34.112 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x8 47/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 48/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-763.ga00c9940.el9. 49/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-nbd-2:19.0.0-763.ga00c9940.el9.x86_64 50/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86 51/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-763.ga00c9940.e 52/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noar 53/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.no 54/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c 55/193 2024-01-20T08:29:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-763.ga00c9940.el9 56/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 57/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-763.ga00c9940.el 58/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 59/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephadm-2:19.0.0-763.ga00c9940.el9.noarch 60/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-top-2:19.0.0-763.ga00c9940.el9.noarch 61/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/193 2024-01-20T08:29:34.114 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 64/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 65/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 67/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 68/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 69/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/193 2024-01-20T08:29:34.115 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 74/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 75/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 76/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 77/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-5.4.4-4.el9.x86_64 78/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 79/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 80/193 2024-01-20T08:29:34.116 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 81/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-13.11-1.el9.x86_64 82/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 83/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 84/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 85/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 86/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 87/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 88/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 89/193 2024-01-20T08:29:34.117 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 90/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 91/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 92/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 93/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 95/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 96/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 97/193 2024-01-20T08:29:34.118 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 98/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 99/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 100/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 101/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 102/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 103/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 104/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 105/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 106/193 2024-01-20T08:29:34.119 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 108/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 109/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : jq-1.6-16.el9.x86_64 110/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 111/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 112/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 113/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 114/193 2024-01-20T08:29:34.120 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 115/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ply-3.11-14.el9.noarch 116/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 117/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 118/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 119/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 120/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 121/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 122/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 123/193 2024-01-20T08:29:34.121 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 124/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 125/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 127/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 128/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 129/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 130/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 131/193 2024-01-20T08:29:34.122 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 132/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 133/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 134/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 135/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 136/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 137/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 138/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 139/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 140/193 2024-01-20T08:29:34.123 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 141/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 142/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 143/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 144/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 145/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 146/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 147/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 148/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 149/193 2024-01-20T08:29:34.124 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 150/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 151/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 152/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 153/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 154/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 155/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 156/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/193 2024-01-20T08:29:34.125 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 158/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : re2-1:20211101-3.el9.x86_64 160/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 161/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:19.0.0-763.ga00c9940.el9.x86_64 162/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 163/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 164/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 165/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 166/193 2024-01-20T08:29:34.126 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 167/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 168/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 169/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 170/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 171/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 172/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 174/193 2024-01-20T08:29:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : acl-2.3.1-4.el9.x86_64 176/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : acl-2.3.1-3.el9.x86_64 177/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 178/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 180/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 181/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 182/193 2024-01-20T08:29:34.128 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 183/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 184/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 185/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 190/193 2024-01-20T08:29:34.129 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 191/193 2024-01-20T08:29:34.130 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 192/193 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 193/193 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:Upgraded: 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: acl-2.3.1-4.el9.x86_64 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-20T08:29:36.902 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-20T08:29:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: automake-1.16.2-8.el9.noarch 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: bison-3.7.4-5.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.905 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.906 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top-2:19.0.0-763.ga00c9940.el9.noarch 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: flex-2.6.4-9.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: jq-1.6-16.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.907 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.908 INFO:teuthology.orchestra.run.smithi116.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: lua-5.4.4-4.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-13.11-1.el9.x86_64 2024-01-20T08:29:36.909 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-20T08:29:36.910 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-20T08:29:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply-3.11-14.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-20T08:29:36.912 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.913 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.914 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-debuginfo-2:19.0.0-763.ga00c9940.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: re2-1:20211101-3.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-20T08:29:36.915 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-20T08:29:36.916 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-01-20T08:29:37.567 DEBUG:teuthology.parallel:result is None 2024-01-20T08:29:37.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:29:37.716 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-20T08:29:37.742 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-763.ga00c9940.el9 2024-01-20T08:29:37.743 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-763.ga00c9940.el9 2024-01-20T08:29:37.743 INFO:teuthology.task.install:The correct ceph version 19.0.0-763.ga00c9940 is installed. 2024-01-20T08:29:37.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:29:37.885 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-20T08:29:37.907 INFO:teuthology.orchestra.run.smithi116.stdout:19.0.0-763.ga00c9940.el9 2024-01-20T08:29:37.907 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-763.ga00c9940.el9 2024-01-20T08:29:37.907 INFO:teuthology.task.install:The correct ceph version 19.0.0-763.ga00c9940 is installed. 2024-01-20T08:29:37.909 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-20T08:29:37.909 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:37.909 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-20T08:29:37.943 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:37.944 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-20T08:29:37.976 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-20T08:29:37.977 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:37.977 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-01-20T08:29:38.014 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-20T08:29:38.083 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:38.083 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2024-01-20T08:29:38.111 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-20T08:29:38.177 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-20T08:29:38.178 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:38.178 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-20T08:29:38.208 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-20T08:29:38.276 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:38.276 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-20T08:29:38.303 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-20T08:29:38.370 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-20T08:29:38.370 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:38.371 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-01-20T08:29:38.400 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-20T08:29:38.469 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:38.470 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/stdin-killer 2024-01-20T08:29:38.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-20T08:29:38.563 INFO:teuthology.run_tasks:Running task ceph... 2024-01-20T08:29:38.641 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-20T08:29:38.641 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2024-01-20T08:29:38.643 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /var/log/ceph 2024-01-20T08:29:38.672 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-20T08:29:38.672 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-20T08:29:38.714 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-20T08:29:38.743 INFO:tasks.ceph:Creating extra log directories... 2024-01-20T08:29:38.744 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-20T08:29:38.786 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-20T08:29:38.820 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-20T08:29:38.820 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-20T08:29:38.821 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-20T08:29:38.821 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:38.821 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-20T08:29:38.871 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-20T08:29:38.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-20T08:29:39.017 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:39.017 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-20T08:29:39.032 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-20T08:29:39.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-20T08:29:39.183 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-20T08:29:39.184 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-20T08:29:39.184 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433', 'branch': 'main', 'description': 'fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7522433', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 450, 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGB32BTEDO0f4g3p4Su/WnQlZgOmLrecAJhuA4qWv3Hx1cNDmUiiybUgT27c7A5Yy4fY0ICXZmnZzUdRNPO3Oio=', 'smithi116.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE16rhgvJXoLMH67byLqrLa9Gu1eNFmFGCvaBgtCERHmUQ1R4dGsq0qngj1/LXWBVgR6VFpcqXa3CROxwDuOR1k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}, 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'workunit': {'clients': {'all': ['suites/dbench.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-19_15:24:44', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-20T08:29:39.184 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-20T08:29:39.187 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-20T08:29:39.234 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2024-01-20T08:29:39.242 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/run/ceph 2024-01-20T08:29:39.300 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:39.300 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-01-20T08:29:39.324 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-20T08:29:39.324 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-01-20 08:29:21.159882969 +0000 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-01-20 08:21:33.018794818 +0000 2024-01-20T08:29:39.379 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-01-20 08:21:33.018794818 +0000 2024-01-20T08:29:39.380 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-01-20 08:21:33.018794818 +0000 2024-01-20T08:29:39.380 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-20T08:29:39.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-01-20T08:29:39.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-01-20T08:29:39.446 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00021244 s, 2.4 MB/s 2024-01-20T08:29:39.447 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-20T08:29:39.504 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-01-20T08:29:39.559 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-01-20 08:29:21.160882946 +0000 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-01-20 08:21:33.370786757 +0000 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-01-20 08:21:33.370786757 +0000 2024-01-20T08:29:39.560 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-01-20 08:21:33.370786757 +0000 2024-01-20T08:29:39.560 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-20T08:29:39.625 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-01-20T08:29:39.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-01-20T08:29:39.626 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000207902 s, 2.5 MB/s 2024-01-20T08:29:39.626 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-20T08:29:39.683 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-01-20 08:29:21.160882946 +0000 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-01-20 08:21:33.713778902 +0000 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-01-20 08:21:33.713778902 +0000 2024-01-20T08:29:39.738 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-01-20 08:21:33.713778902 +0000 2024-01-20T08:29:39.738 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-20T08:29:39.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-01-20T08:29:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-01-20T08:29:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000228654 s, 2.2 MB/s 2024-01-20T08:29:39.805 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-20T08:29:39.860 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-01-20 08:29:21.160882946 +0000 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-01-20 08:21:34.056771047 +0000 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-01-20 08:21:34.056771047 +0000 2024-01-20T08:29:39.916 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-01-20 08:21:34.056771047 +0000 2024-01-20T08:29:39.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-20T08:29:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-01-20T08:29:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-01-20T08:29:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000190056 s, 2.7 MB/s 2024-01-20T08:29:39.982 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-20T08:29:40.038 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:40.038 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:40.038 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-01-20T08:29:40.056 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-20T08:29:40.056 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-01-20T08:29:40.113 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-20T08:29:40.113 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:40.113 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-01-20T08:29:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-01-20 08:29:25.620581736 +0000 2024-01-20T08:29:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-01-20 08:21:33.024116678 +0000 2024-01-20T08:29:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-01-20 08:21:33.024116678 +0000 2024-01-20T08:29:40.114 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-01-20 08:21:33.024116678 +0000 2024-01-20T08:29:40.114 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-20T08:29:40.181 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-01-20T08:29:40.182 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-01-20T08:29:40.182 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000215984 s, 2.4 MB/s 2024-01-20T08:29:40.183 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-20T08:29:40.240 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-01-20 08:29:25.620581736 +0000 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-01-20 08:21:33.361110980 +0000 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-01-20 08:21:33.361110980 +0000 2024-01-20T08:29:40.297 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-01-20 08:21:33.361110980 +0000 2024-01-20T08:29:40.298 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-20T08:29:40.363 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-01-20T08:29:40.364 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-01-20T08:29:40.364 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000200938 s, 2.5 MB/s 2024-01-20T08:29:40.365 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-20T08:29:40.421 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-01-20T08:29:40.476 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-01-20 08:29:25.621581719 +0000 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-01-20 08:21:33.701105232 +0000 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-01-20 08:21:33.701105232 +0000 2024-01-20T08:29:40.477 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-01-20 08:21:33.701105232 +0000 2024-01-20T08:29:40.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-20T08:29:40.543 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-01-20T08:29:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-01-20T08:29:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000217378 s, 2.4 MB/s 2024-01-20T08:29:40.785 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-20T08:29:40.844 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-01-20 08:29:25.621581719 +0000 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-01-20 08:21:34.038099535 +0000 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-01-20 08:21:34.038099535 +0000 2024-01-20T08:29:40.900 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-01-20 08:21:34.038099535 +0000 2024-01-20T08:29:40.901 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-20T08:29:40.966 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-01-20T08:29:40.966 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-01-20T08:29:40.966 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.00021865 s, 2.3 MB/s 2024-01-20T08:29:40.967 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-20T08:29:41.025 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:41.025 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-20T08:29:41.026 INFO:tasks.ceph:Generating config... 2024-01-20T08:29:41.027 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-20T08:29:41.027 INFO:tasks.ceph:[client] debug client = 20 2024-01-20T08:29:41.027 INFO:tasks.ceph:[client] debug ms = 1 2024-01-20T08:29:41.027 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-20T08:29:41.027 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-20T08:29:41.027 INFO:tasks.ceph:[global] mds beacon grace = 60 2024-01-20T08:29:41.027 INFO:tasks.ceph:[global] mds heartbeat grace = 60 2024-01-20T08:29:41.027 INFO:tasks.ceph:[global] osd heartbeat grace = 60 2024-01-20T08:29:41.027 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-20T08:29:41.027 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] mds valgrind exit = True 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-20T08:29:41.028 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-20T08:29:41.029 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-20T08:29:41.029 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-20T08:29:41.030 INFO:tasks.ceph:Setting up mon.a... 2024-01-20T08:29:41.030 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-20T08:29:41.096 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.keyring 2024-01-20T08:29:41.099 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-20T08:29:41.146 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-20T08:29:41.174 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.87'), ('mon.b', '172.21.15.116'), ('mon.c', '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]')] 2024-01-20T08:29:41.174 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.87,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]', 'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-20T08:29:41.176 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:41.176 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-20T08:29:41.229 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.87 --add b 172.21.15.116 --addv c '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:41.315 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:41.315 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: generated fsid 2b2378c9-6207-4be4-af26-3b0cbbc1772f 2024-01-20T08:29:41.315 INFO:teuthology.orchestra.run.smithi087.stdout:setting min_mon_release = quincy 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 0 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:fsid 2b2378c9-6207-4be4-af26-3b0cbbc1772f 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:last_changed 2024-01-20T08:29:41.313325+0000 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-01-20T08:29:41.313325+0000 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:min_mon_release 17 (quincy) 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:election_strategy: 1 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:1: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.b 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:2: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-01-20T08:29:41.316 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-20T08:29:41.318 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-20T08:29:41.332 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2b2378c9-6207-4be4-af26-3b0cbbc1772f... 2024-01-20T08:29:41.333 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-20T08:29:41.375 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-20T08:29:41.423 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-01-20T08:29:41.423 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-01-20T08:29:41.423 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-01-20T08:29:41.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-01-20T08:29:41.425 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.426 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]" 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: mds beacon grace = 60 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: mds heartbeat grace = 60 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: osd heartbeat grace = 60 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 2b2378c9-6207-4be4-af26-3b0cbbc1772f 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-01-20T08:29:41.427 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-01-20T08:29:41.428 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-01-20T08:29:41.429 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 20 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 20 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 10 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: osd fast shutdown = False 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-01-20T08:29:41.430 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-01-20T08:29:41.431 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-01-20T08:29:41.432 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd crush smoke test = False 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-01-20T08:29:41.433 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-01-20T08:29:41.434 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.e] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.f] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = True 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: mds valgrind exit = True 2024-01-20T08:29:41.435 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = True 2024-01-20T08:29:41.436 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-01-20T08:29:41.436 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-01-20T08:29:41.436 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-01-20T08:29:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-01-20T08:29:41.436 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-01-20T08:29:41.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-01-20T08:29:41.438 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.87,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]" 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds beacon grace = 60 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds heartbeat grace = 60 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: osd heartbeat grace = 60 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 2b2378c9-6207-4be4-af26-3b0cbbc1772f 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-01-20T08:29:41.440 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-01-20T08:29:41.442 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: osd fast shutdown = False 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.443 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-01-20T08:29:41.444 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-20T08:29:41.445 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd crush smoke test = False 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-01-20T08:29:41.446 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.e] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-01-20T08:29:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.f] 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: mds valgrind exit = True 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-01-20T08:29:41.448 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-01-20T08:29:41.450 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-20T08:29:41.450 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-20T08:29:41.545 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-20T08:29:41.546 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:41.546 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-20T08:29:41.561 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:41.561 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-20T08:29:41.615 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:29:41.615 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:41.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-20T08:29:41.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-20T08:29:41.694 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:29:41.694 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:41.749 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T08:29:41.749 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:41.749 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-20T08:29:41.749 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-20T08:29:41.792 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:29:41.792 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:41.849 INFO:tasks.ceph:Setting up mon nodes... 2024-01-20T08:29:41.849 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-20T08:29:41.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-20T08:29:41.903 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-20T08:29:41.905 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-20T08:29:41.969 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-20T08:29:41.972 INFO:tasks.ceph:Setting up mds nodes... 2024-01-20T08:29:41.972 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-20T08:29:42.026 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-20T08:29:42.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-20T08:29:42.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-20T08:29:42.150 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-20T08:29:42.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-20T08:29:42.178 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-01-20T08:29:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-01-20T08:29:42.275 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-01-20T08:29:42.302 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-20T08:29:42.361 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-20T08:29:42.363 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-20T08:29:42.390 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-20T08:29:42.485 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-20T08:29:42.488 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-20T08:29:42.514 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-01-20T08:29:42.608 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-01-20T08:29:42.610 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-01-20T08:29:42.636 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-20T08:29:42.637 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-20T08:29:42.679 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-20T08:29:42.694 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-20T08:29:42.694 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-20T08:29:42.695 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-20T08:29:42.761 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:42.762 INFO:tasks.ceph:role: osd.0 2024-01-20T08:29:42.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:29:42.762 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-20T08:29:42.835 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:42.835 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:42.836 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:43.356 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-01-20T08:29:43.358 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-01-20T08:29:43.359 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-20T08:29:43.521 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-20T08:29:43.557 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-20T08:29:43.624 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:43.624 INFO:tasks.ceph:role: osd.1 2024-01-20T08:29:43.625 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:29:43.625 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-20T08:29:43.699 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:43.699 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:43.699 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:43.699 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:43.699 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:43.700 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:43.700 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:43.700 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:43.700 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:43.700 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:44.164 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-01-20T08:29:44.166 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-01-20T08:29:44.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-20T08:29:44.260 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-20T08:29:44.294 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-20T08:29:44.361 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:44.361 INFO:tasks.ceph:role: osd.2 2024-01-20T08:29:44.361 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:29:44.361 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-20T08:29:44.434 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:44.434 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:44.434 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:44.434 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:44.434 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:44.435 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:44.435 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:44.435 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:44.435 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:44.435 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:44.920 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-01-20T08:29:44.922 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-01-20T08:29:44.923 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-20T08:29:45.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-20T08:29:45.086 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-20T08:29:45.153 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:45.153 INFO:tasks.ceph:role: osd.3 2024-01-20T08:29:45.153 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:29:45.153 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:45.228 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:45.736 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-01-20T08:29:45.737 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-01-20T08:29:45.737 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-20T08:29:45.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-20T08:29:45.896 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:46.002+0000 7fe004128640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-20T08:29:46.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:46.003+0000 7fe004128640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-20T08:29:46.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:46.003+0000 7fe004128640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-20T08:29:46.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:46.003+0000 7fe004128640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-20T08:29:46.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:46.003+0000 7fe004128640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-20T08:29:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:46.003+0000 7fe004128640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-20T08:29:48.856 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-20T08:29:48.888 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:48.976+0000 7f38f8c0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-20T08:29:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:48.976+0000 7f38f8c0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-20T08:29:48.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:48.976+0000 7f38f8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-20T08:29:48.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:48.976+0000 7f38f8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-20T08:29:48.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:48.976+0000 7f38f8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-20T08:29:48.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:48.977+0000 7f38f8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-20T08:29:51.830 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-20T08:29:51.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:51.948+0000 7ff47e32a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-20T08:29:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:51.948+0000 7ff47e32a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-20T08:29:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:51.948+0000 7ff47e32a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-20T08:29:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:51.948+0000 7ff47e32a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-20T08:29:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:51.948+0000 7ff47e32a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-20T08:29:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:51.949+0000 7ff47e32a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-20T08:29:54.807 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-20T08:29:54.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:29:54.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:54.921+0000 7f59a276f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-20T08:29:54.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:54.921+0000 7f59a276f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-20T08:29:54.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:54.922+0000 7f59a276f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-20T08:29:54.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:54.922+0000 7f59a276f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-20T08:29:54.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:54.922+0000 7f59a276f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-20T08:29:54.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:29:54.922+0000 7f59a276f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-20T08:29:57.774 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-20T08:29:57.805 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-20T08:29:57.837 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:57.837 INFO:tasks.ceph:role: osd.4 2024-01-20T08:29:57.837 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T08:29:57.837 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-20T08:29:57.915 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:57.915 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:57.915 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:57.915 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:57.915 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:57.916 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:57.916 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:57.916 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:57.916 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:57.916 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:58.422 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-01-20T08:29:58.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-01-20T08:29:58.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-20T08:29:58.556 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-20T08:29:58.588 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-20T08:29:58.656 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:58.656 INFO:tasks.ceph:role: osd.5 2024-01-20T08:29:58.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T08:29:58.656 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:58.731 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:59.178 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-01-20T08:29:59.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-01-20T08:29:59.180 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-20T08:29:59.273 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-20T08:29:59.303 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-20T08:29:59.370 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-20T08:29:59.371 INFO:tasks.ceph:role: osd.6 2024-01-20T08:29:59.371 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T08:29:59.371 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:29:59.445 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:29:59.923 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-01-20T08:29:59.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-01-20T08:29:59.925 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-20T08:30:00.073 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-20T08:30:00.105 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-20T08:30:00.173 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-20T08:30:00.173 INFO:tasks.ceph:role: osd.7 2024-01-20T08:30:00.173 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T08:30:00.173 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-20T08:30:00.247 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-20T08:30:00.247 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-20T08:30:00.247 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-20T08:30:00.248 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-20T08:30:00.766 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-01-20T08:30:00.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-01-20T08:30:00.768 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-20T08:30:00.900 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-20T08:30:00.932 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:30:01.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:01.023+0000 7fe08e8b3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-20T08:30:01.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:01.023+0000 7fe08e8b3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-20T08:30:01.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:01.023+0000 7fe08e8b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-20T08:30:01.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:01.023+0000 7fe08e8b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-20T08:30:01.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:01.023+0000 7fe08e8b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-20T08:30:01.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:01.023+0000 7fe08e8b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-20T08:30:03.880 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-20T08:30:03.909 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:30:03.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:03.994+0000 7fae66408640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-20T08:30:03.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:03.994+0000 7fae66408640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-20T08:30:03.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:03.994+0000 7fae66408640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-20T08:30:03.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:03.994+0000 7fae66408640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-20T08:30:03.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:03.994+0000 7fae66408640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-20T08:30:03.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:03.995+0000 7fae66408640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-20T08:30:06.839 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-20T08:30:06.868 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:30:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:06.953+0000 7fa0921ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-20T08:30:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:06.953+0000 7fa0921ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-20T08:30:06.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:06.953+0000 7fa0921ac640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-20T08:30:06.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:06.953+0000 7fa0921ac640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-20T08:30:06.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:06.953+0000 7fa0921ac640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-20T08:30:06.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:06.954+0000 7fa0921ac640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-20T08:30:09.186 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:30:09.189 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:30:09.805 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-20T08:30:09.834 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:30:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:09.919+0000 7fca7e6f9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-20T08:30:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:09.919+0000 7fca7e6f9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-20T08:30:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:09.919+0000 7fca7e6f9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-20T08:30:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:09.919+0000 7fca7e6f9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-20T08:30:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:09.919+0000 7fca7e6f9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-20T08:30:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:09.920+0000 7fca7e6f9640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-20T08:30:12.773 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-20T08:30:12.804 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-20T08:30:12.804 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:12.804 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-20T08:30:12.836 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:12.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-20T08:30:12.906 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:12.907 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-20T08:30:12.976 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:12.976 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-01-20T08:30:13.044 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:13.044 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-20T08:30:13.111 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:13.111 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-20T08:30:13.179 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:13.179 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-20T08:30:13.245 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:13.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-20T08:30:13.312 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.313 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-20T08:30:13.345 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.345 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-20T08:30:13.416 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-20T08:30:13.485 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.486 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-01-20T08:30:13.554 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.554 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-20T08:30:13.623 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.624 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-20T08:30:13.692 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.692 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-20T08:30:13.759 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:13.759 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-20T08:30:13.825 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:13.825 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-20T08:30:13.841 INFO:tasks.ceph:Adding keys to all mons... 2024-01-20T08:30:13.841 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-20T08:30:13.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB1hKtlKUu4NRAAqBwMp8vJuMVkx3hbzSaEsg== 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlnOp1ARAA1ZMGa9PCyPe6VaSv7r/CDQ== 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlKtTVCBAA7yY1wpQ3rbzP8JaOGCQZig== 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.e] 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlN1wvEBAA7u/xxBTgncWHYy9AQfErrQ== 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB6hKtl0ZUxABAAjl+/OHVyezJojOu2UUXjpg== 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2024-01-20T08:30:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB8hKtlgYs+OhAA+SXpMeBnhqAvQHbHNCwTyw== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB/hKtl/zaXOBAArg5T/YAqeMe6quCjXpujSg== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQCChKtlvOn8NhAAFs+vth2TjRtnPFt6fU/Srw== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB1hKtlQxGwORAADqoq3L5ex4vGjISQs4JQiw== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlM2trFRAA/RfpuQBk9qAbAy117yGPEw== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlrc7SHBAASU1tNrB23LPX2VdHriXapw== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.f] 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlNU0oJBAAQN1ZAnd8Gm7WrV3G48h9qA== 2024-01-20T08:30:13.907 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQCJhKtl2BB9ARAAlSr/R6JED03dncBISUiu7A== 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQCLhKtlqXJjOxAAwW+aIJjvZ65nAPiN7dWc5A== 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQCOhKtlVrfgOBAANG6XPYFyzFaf2vMa6W0d9w== 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQCRhKtlJm7ZNhAAd6cbU03Tha44EUhQ07uYgg== 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-01-20T08:30:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQB2hKtlTU9jKBAASbiB/X7N22u3qLdDXnlEkg== 2024-01-20T08:30:13.909 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2024-01-20T08:30:13.909 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB1hKtlKUu4NRAAqBwMp8vJuMVkx3hbzSaEsg== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlnOp1ARAA1ZMGa9PCyPe6VaSv7r/CDQ== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlKtTVCBAA7yY1wpQ3rbzP8JaOGCQZig== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.e] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlN1wvEBAA7u/xxBTgncWHYy9AQfErrQ== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB6hKtl0ZUxABAAjl+/OHVyezJojOu2UUXjpg== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB8hKtlgYs+OhAA+SXpMeBnhqAvQHbHNCwTyw== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB/hKtl/zaXOBAArg5T/YAqeMe6quCjXpujSg== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCChKtlvOn8NhAAFs+vth2TjRtnPFt6fU/Srw== 2024-01-20T08:30:13.910 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.x] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB1hKtlQxGwORAADqoq3L5ex4vGjISQs4JQiw== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlM2trFRAA/RfpuQBk9qAbAy117yGPEw== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlrc7SHBAASU1tNrB23LPX2VdHriXapw== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.f] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlNU0oJBAAQN1ZAnd8Gm7WrV3G48h9qA== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.4] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCJhKtl2BB9ARAAlSr/R6JED03dncBISUiu7A== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.5] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCLhKtlqXJjOxAAwW+aIJjvZ65nAPiN7dWc5A== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.6] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCOhKtlVrfgOBAANG6XPYFyzFaf2vMa6W0d9w== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.7] 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCRhKtlJm7ZNhAAd6cbU03Tha44EUhQ07uYgg== 2024-01-20T08:30:13.911 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-01-20T08:30:13.912 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQB2hKtlTU9jKBAASbiB/X7N22u3qLdDXnlEkg== 2024-01-20T08:30:13.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-20T08:30:13.952 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-20T08:30:14.003 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.060 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.062 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.117 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.119 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.173 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.175 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.229 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.283 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.334 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.336 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.388 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-20T08:30:14.390 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-20T08:30:14.441 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.443 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.498 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.555 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-20T08:30:14.609 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.611 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.665 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.719 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.772 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-20T08:30:14.824 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-20T08:30:14.826 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-20T08:30:14.880 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-20T08:30:14.881 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-20T08:30:14.907 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-20T08:30:15.002 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-20T08:30:15.028 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-20T08:30:15.055 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-20T08:30:15.151 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-20T08:30:15.178 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-20T08:30:15.244 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-20T08:30:15.340 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-20T08:30:15.367 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:30:15.369 DEBUG:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-20T08:30:15.421 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-20T08:30:15.421 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-20T08:30:15.421 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-20T08:30:15.422 DEBUG:teuthology.orchestra.run.smithi087:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2024-01-20T08:30:15.424 INFO:tasks.ceph.mon.a:Started 2024-01-20T08:30:15.424 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-20T08:30:15.424 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-20T08:30:15.424 DEBUG:teuthology.orchestra.run.smithi116:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2024-01-20T08:30:15.463 INFO:tasks.ceph.mon.b:Started 2024-01-20T08:30:15.463 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-20T08:30:15.463 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-20T08:30:15.463 DEBUG:teuthology.orchestra.run.smithi116:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2024-01-20T08:30:15.466 INFO:tasks.ceph.mon.c:Started 2024-01-20T08:30:15.466 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-20T08:30:15.466 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-20T08:30:15.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-01-20T08:30:15.468 INFO:tasks.ceph.mgr.y:Started 2024-01-20T08:30:15.468 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-20T08:30:15.468 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-01-20T08:30:15.470 INFO:tasks.ceph.mgr.x:Started 2024-01-20T08:30:15.470 INFO:tasks.ceph:Setting crush tunables to default 2024-01-20T08:30:15.470 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd crush tunables default 2024-01-20T08:30:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 Processor -- start 2024-01-20T08:30:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 -- start start 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 0x7fdbe40c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 0x7fdbe406f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe4156d30 con 0x7fdbe406e3f0 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbe41571c0 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.568+0000 7fdbeadde640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe406ecf0 con 0x7fdbe4150730 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.569+0000 7fdbe3fff640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 msgr2=0x7fdbe406f890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.569+0000 7fdbe8b7a640 1 -- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 msgr2=0x7fdbe40c6860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3301/0 2024-01-20T08:30:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.569+0000 7fdbe3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 0x7fdbe406f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:15.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.569+0000 7fdbe8b7a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 0x7fdbe40c6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:15.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.569+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbe406e3f0 legacy=0x7fdbe40c88f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:15.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.770+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbe406e3f0 legacy=0x7fdbe40c88f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:15.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.770+0000 7fdbe8b7a640 1 -- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 msgr2=0x7fdbe40c6860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3301/0 2024-01-20T08:30:15.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.770+0000 7fdbe3fff640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 msgr2=0x7fdbe406f890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:15.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.770+0000 7fdbe8b7a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 0x7fdbe40c6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:15.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:15.770+0000 7fdbe3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 0x7fdbe406f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:16.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.171+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbe406e3f0 legacy=0x7fdbe40c88f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.171+0000 7fdbe8b7a640 1 -- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 msgr2=0x7fdbe40c6860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3301/0 2024-01-20T08:30:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.171+0000 7fdbe3fff640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 msgr2=0x7fdbe406f890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.171+0000 7fdbe8b7a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 0x7fdbe40c6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.171+0000 7fdbe3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 0x7fdbe406f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:16.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.972+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbe406e3f0 legacy=0x7fdbe40c88f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:16.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.972+0000 7fdbe8b7a640 1 -- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 msgr2=0x7fdbe40c6860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3301/0 2024-01-20T08:30:16.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.972+0000 7fdbe3fff640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 msgr2=0x7fdbe406f890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:16.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.972+0000 7fdbe8b7a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 0x7fdbe40c6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:16.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:16.972+0000 7fdbe3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 0x7fdbe406f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:18.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 -- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 msgr2=0x7fdbe40c6860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:18.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbe4150730 0x7fdbe40c6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 msgr2=0x7fdbe406f890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbe41571c0 0x7fdbe406f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbe406e3f0 legacy=0x7fdbe40c88f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 0x7fdbc8005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe406efd0 con 0x7fdbc8006220 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe27fc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbc80036c0 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.87:3300/0 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:18.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.569+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc8006220 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:18.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.770+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.87:3300/0 2024-01-20T08:30:18.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.770+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:18.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:18.770+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc8006220 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:19.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:19.171+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.87:3300/0 2024-01-20T08:30:19.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:19.171+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:19.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:19.171+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc8006220 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:19.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:19.972+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.87:3300/0 2024-01-20T08:30:19.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:19.972+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:19.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:19.972+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc8006220 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 -- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc80036c0 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc8006220 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe937b640 1 -- reap_dead start 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe4156d30 con 0x7fdbc800a850 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe406efd0 con 0x7fdbc800a360 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe27fc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbc8009f80 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbc800a850 legacy=0x7fdbc800d0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc800a360 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:21.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.569+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:21.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.770+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbc800a850 legacy=0x7fdbc800d0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:21.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.770+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:21.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.770+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc800a360 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:21.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:21.770+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.171+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbc800a850 legacy=0x7fdbc800d0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:22.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.171+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:22.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.171+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc800a360 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:22.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.171+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:22.309 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:00:06.786 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:30:22.405 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:00:06.839 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:30:22.410 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:00:06.844 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:30:22.617 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:00:07.093 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:30:22.724 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:00:07.157 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:30:22.729 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:00:07.162 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:30:22.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.972+0000 7fdbe937b640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbc800a850 legacy=0x7fdbc800d0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.87:6789/0 2024-01-20T08:30:22.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.972+0000 7fdbe8b7a640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.116:3300/0 2024-01-20T08:30:22.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.972+0000 7fdbe3fff640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc800a360 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.116:6789/0 2024-01-20T08:30:22.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:22.972+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:24.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 -- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:24.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:24.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 -- >> v1:172.21.15.116:6789/0 conn(0x7fdbc800a360 legacy=0x7fdbc80086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:24.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 -- >> v1:172.21.15.87:6789/0 conn(0x7fdbc800a850 legacy=0x7fdbc800d0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:24.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:24.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:24.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbc800e880 2024-01-20T08:30:24.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.569+0000 7fdbe27fc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe406ecf0 con 0x7fdbc800f300 2024-01-20T08:30:24.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.760+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:24.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.762+0000 7fdbe8b7a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37594/0 (socket says 172.21.15.87:37594) 2024-01-20T08:30:24.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.762+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 learned_addr learned my addr 172.21.15.87:0/2918556441 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:24.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.809+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:24.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.813+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-20T08:30:24.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.813+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:24.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.813+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:24.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.813+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:24.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.863+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:24.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.863+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:24.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.863+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:24.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:24.863+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:25.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.064+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.065+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-20T08:30:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.065+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.065+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.065+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:25.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.125+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:25.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.160+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:25.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.160+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:25.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.160+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:25.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.160+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:25.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.528+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.530+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.530+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.530+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.530+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:25.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.565+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:25.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.565+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:25.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.565+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:25.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.565+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:25.566+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:26.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.333+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:26.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.334+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:26.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.334+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:26.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.334+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:26.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.334+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.367+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:26.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.368+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:26.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.368+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:26.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.368+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:26.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:26.368+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 msgr2=0x7fdbc800f730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800f300 0x7fdbc800f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 msgr2=0x7fdbc800ec90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800e880 0x7fdbc800ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe937b640 1 -- 172.21.15.87:0/2918556441 reap_dead start 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc8010870 con 0x7fdbc800a850 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbc800a360 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.569+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe406ecf0 con 0x7fdbc8009f80 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.570+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.570+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.571+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.571+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.571+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.571+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.579+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.579+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.579+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.579+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.772+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.773+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.773+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.773+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.773+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:27.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.780+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.781+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.781+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.781+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:27.782+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:28.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.174+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:28.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.175+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:28.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.175+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:28.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.175+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:28.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.175+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:28.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.183+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:28.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.185+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:28.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.185+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:28.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.185+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:28.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.185+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:28.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.977+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:28.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.978+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:28.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.978+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:28.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.978+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:28.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:28.978+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:29.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:29.000+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:29.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:29.001+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:29.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:29.001+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:29.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:29.001+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:29.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:29.001+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 msgr2=0x7fdbc8005ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.116:3300/0 conn(0x7fdbc8009f80 0x7fdbc8005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 msgr2=0x7fdbc800ec60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> v2:172.21.15.87:3300/0 conn(0x7fdbc800a360 0x7fdbc800ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 >> v1:172.21.15.116:6789/0 conn(0x7fdbc800a850 legacy=0x7fdbc8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe4156d30 con 0x7fdbc800d0f0 2024-01-20T08:30:30.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.570+0000 7fdbe27fc640 1 -- 172.21.15.87:0/2918556441 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbc800fec0 2024-01-20T08:30:30.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.571+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:30.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.572+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:30.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.572+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:30.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.572+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:30.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.572+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-20T08:30:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.773+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.774+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.774+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.774+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:30.774+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-20T08:30:31.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.175+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:31.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.176+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:31.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.176+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:31.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.176+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:31.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.177+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-20T08:30:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.978+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.979+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-20T08:30:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.979+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-20T08:30:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.979+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-20T08:30:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:31.979+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-20T08:30:32.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.602+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747441429 0 0) 0x7fdbe4156d30 con 0x7fdbc800d0f0 2024-01-20T08:30:32.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.602+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd4003520 con 0x7fdbc800d0f0 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644747953 0 0) 0x7fdbd4003520 con 0x7fdbc800d0f0 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 msgr2=0x7fdbc80102a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbc800fec0 0x7fdbc80102a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe406efd0 con 0x7fdbc800d0f0 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4017552009 0 0) 0x7fdbd0014350 con 0x7fdbc800d0f0 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 >> v1:172.21.15.87:6789/0 conn(0x7fdbc800d0f0 legacy=0x7fdbc800d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe937b640 1 -- 172.21.15.87:0/2918556441 reap_dead start 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40039e0 0x7fdbd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40047c0 0x7fdbd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40052a0 0x7fdbd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe4156460 con 0x7fdbd40047c0 2024-01-20T08:30:32.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd400c170 con 0x7fdbd40039e0 2024-01-20T08:30:32.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.647+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd400c330 con 0x7fdbd40052a0 2024-01-20T08:30:32.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.648+0000 7fdbe937b640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40052a0 0x7fdbd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:32.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.648+0000 7fdbe8b7a640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40039e0 0x7fdbd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:32.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.666+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40047c0 0x7fdbd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.679+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40052a0 msgr2=0x7fdbd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.679+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40052a0 0x7fdbd400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.679+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40039e0 msgr2=0x7fdbd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.679+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40039e0 0x7fdbd4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.679+0000 7fdbe3fff640 1 -- 172.21.15.87:0/2918556441 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4003520 con 0x7fdbd40047c0 2024-01-20T08:30:32.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.686+0000 7fdbe3fff640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40047c0 0x7fdbd4004c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0008010 tx=0x7fdbd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d874bbd44ddf7772 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:32.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.716+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0035440 con 0x7fdbd40047c0 2024-01-20T08:30:32.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.716+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd00355a0 con 0x7fdbd40047c0 2024-01-20T08:30:32.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.716+0000 7fdbe37fe640 1 -- 172.21.15.87:0/2918556441 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0035950 con 0x7fdbd40047c0 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.716+0000 7fdbeadde640 1 -- 172.21.15.87:0/2918556441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40047c0 msgr2=0x7fdbd4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.716+0000 7fdbeadde640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40047c0 0x7fdbd4004c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0008010 tx=0x7fdbd40057f0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 -- 172.21.15.87:0/2918556441 shutdown_connections 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40052a0 0x7fdbd400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40047c0 0x7fdbd4004c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 --2- 172.21.15.87:0/2918556441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40039e0 0x7fdbd4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 -- 172.21.15.87:0/2918556441 >> 172.21.15.87:0/2918556441 conn(0x7fdbe406b6c0 msgr2=0x7fdbe406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 -- 172.21.15.87:0/2918556441 shutdown_connections 2024-01-20T08:30:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.717+0000 7fdbeadde640 1 -- 172.21.15.87:0/2918556441 wait complete. 2024-01-20T08:30:32.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.718+0000 7fdbeadde640 1 Processor -- start 2024-01-20T08:30:32.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.718+0000 7fdbeadde640 1 -- start start 2024-01-20T08:30:32.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbeadde640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40039e0 0x7fdbe414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:32.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbeadde640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40047c0 0x7fdbe41f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:32.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbeadde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40052a0 0x7fdbe40c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:32.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbeadde640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe41e29c0 con 0x7fdbd40052a0 2024-01-20T08:30:32.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbeadde640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe41f2470 con 0x7fdbd40047c0 2024-01-20T08:30:32.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbeadde640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40c3650 con 0x7fdbd40039e0 2024-01-20T08:30:32.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbe3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40047c0 0x7fdbe41f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:32.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.719+0000 7fdbe8b7a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40039e0 0x7fdbe414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:32.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.720+0000 7fdbe8b7a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40039e0 0x7fdbe414f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:51440/0 (socket says 172.21.15.87:51440) 2024-01-20T08:30:32.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.720+0000 7fdbe8b7a640 1 -- 172.21.15.87:0/3510139206 learned_addr learned my addr 172.21.15.87:0/3510139206 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:32.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.720+0000 7fdbe937b640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40052a0 0x7fdbe40c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:32.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.732+0000 7fdbe937b640 1 -- 172.21.15.87:0/3510139206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40039e0 msgr2=0x7fdbe414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:32.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.732+0000 7fdbe937b640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40039e0 0x7fdbe414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.732+0000 7fdbe937b640 1 -- 172.21.15.87:0/3510139206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40047c0 msgr2=0x7fdbe41f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:32.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.732+0000 7fdbe937b640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40047c0 0x7fdbe41f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:32.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.732+0000 7fdbe937b640 1 -- 172.21.15.87:0/3510139206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe41f1df0 con 0x7fdbd40052a0 2024-01-20T08:30:32.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.734+0000 7fdbe937b640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40052a0 0x7fdbe40c3110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdbe406d780 tx=0x7fdbdc00fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:32.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.738+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00da00 con 0x7fdbd40052a0 2024-01-20T08:30:32.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.739+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc008040 con 0x7fdbd40052a0 2024-01-20T08:30:32.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.739+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00dbc0 con 0x7fdbd40052a0 2024-01-20T08:30:32.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.739+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe41e1bf0 con 0x7fdbd40052a0 2024-01-20T08:30:32.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.739+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe41f2090 con 0x7fdbd40052a0 2024-01-20T08:30:32.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.740+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe41f25b0 con 0x7fdbd40052a0 2024-01-20T08:30:32.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.912+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fdbdc0227b0 con 0x7fdbd40052a0 2024-01-20T08:30:32.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.912+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdbdc019ab0 con 0x7fdbd40052a0 2024-01-20T08:30:32.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.912+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fdbe41f25b0 con 0x7fdbd40052a0 2024-01-20T08:30:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:32.974+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdbe4074290 con 0x7fdbd40052a0 2024-01-20T08:30:33.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.080+0000 7fdbe17fa640 1 -- 172.21.15.87:0/3510139206 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdbdc026020 con 0x7fdbd40052a0 2024-01-20T08:30:33.083 INFO:teuthology.orchestra.run.smithi087.stderr:adjusted tunables profile to default 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40052a0 msgr2=0x7fdbe40c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40052a0 0x7fdbe40c3110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdbe406d780 tx=0x7fdbdc00fd70 comp rx=0 tx=0).stop 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 shutdown_connections 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd40052a0 0x7fdbe40c3110 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbd40047c0 0x7fdbe41f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 --2- 172.21.15.87:0/3510139206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbd40039e0 0x7fdbe414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 >> 172.21.15.87:0/3510139206 conn(0x7fdbe406b6c0 msgr2=0x7fdbe40c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 shutdown_connections 2024-01-20T08:30:33.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:33.081+0000 7fdbeadde640 1 -- 172.21.15.87:0/3510139206 wait complete. 2024-01-20T08:30:33.094 INFO:tasks.ceph:check_enable_crimson: False 2024-01-20T08:30:33.094 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-20T08:30:33.095 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:33.095 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-20T08:30:33.121 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:33.121 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-20T08:30:33.188 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:33.189 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-20T08:30:33.255 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:30:33.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-20T08:30:33.321 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:33.321 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-20T08:30:33.349 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:33.349 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-20T08:30:33.415 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:33.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-20T08:30:33.482 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-01-20T08:30:33.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-20T08:30:33.550 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 59b49896-16a3-4471-834a-3f9977004332 0 2024-01-20T08:30:33.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.692+0000 7f911f0e7640 1 Processor -- start 2024-01-20T08:30:33.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.692+0000 7f911f0e7640 1 -- start start 2024-01-20T08:30:33.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911f0e7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91181506f0 0x7f911815a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911f0e7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91180c0f20 con 0x7f9118157920 2024-01-20T08:30:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911f0e7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9118156da0 con 0x7f9118156430 2024-01-20T08:30:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911f0e7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180c91c0 con 0x7f91181506f0 2024-01-20T08:30:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911d684640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9118156430 0x7f911806fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49942/0 (socket says 172.21.15.116:49942) 2024-01-20T08:30:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911d684640 1 -- 172.21.15.116:0/1569003494 learned_addr learned my addr 172.21.15.116:0/1569003494 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.693+0000 7f911ce83640 1 --2- 172.21.15.116:0/1569003494 >> v2:172.21.15.87:3300/0 conn(0x7f91181506f0 0x7f911815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.696+0000 7f911ce83640 1 -- 172.21.15.116:0/1569003494 >> v1:172.21.15.116:6789/0 conn(0x7f9118156430 legacy=0x7f911806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.696+0000 7f911ce83640 1 -- 172.21.15.116:0/1569003494 >> v1:172.21.15.87:6789/0 conn(0x7f9118157920 legacy=0x7f91180c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.696+0000 7f911ce83640 1 -- 172.21.15.116:0/1569003494 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118157060 con 0x7f91181506f0 2024-01-20T08:30:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.702+0000 7f911ce83640 1 --2- 172.21.15.116:0/1569003494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91181506f0 0x7f911815a450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f910c0097a0 tx=0x7f910c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9d0d748f4aeb50 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.706+0000 7f91177fe640 1 -- 172.21.15.116:0/1569003494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c031830 con 0x7f91181506f0 2024-01-20T08:30:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.706+0000 7f91177fe640 1 -- 172.21.15.116:0/1569003494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c031990 con 0x7f91181506f0 2024-01-20T08:30:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.706+0000 7f91177fe640 1 -- 172.21.15.116:0/1569003494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c031d20 con 0x7f91181506f0 2024-01-20T08:30:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 -- 172.21.15.116:0/1569003494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91181506f0 msgr2=0x7f911815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 --2- 172.21.15.116:0/1569003494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91181506f0 0x7f911815a450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f910c0097a0 tx=0x7f910c02b8d0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 -- 172.21.15.116:0/1569003494 shutdown_connections 2024-01-20T08:30:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 --2- 172.21.15.116:0/1569003494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91181506f0 0x7f911815a450 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 -- 172.21.15.116:0/1569003494 >> 172.21.15.116:0/1569003494 conn(0x7f9118072810 msgr2=0x7f911806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 -- 172.21.15.116:0/1569003494 shutdown_connections 2024-01-20T08:30:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.707+0000 7f911f0e7640 1 -- 172.21.15.116:0/1569003494 wait complete. 2024-01-20T08:30:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.709+0000 7f911f0e7640 1 Processor -- start 2024-01-20T08:30:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.709+0000 7f911f0e7640 1 -- start start 2024-01-20T08:30:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.710+0000 7f911f0e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91181506f0 0x7f911814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.710+0000 7f911f0e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9118156430 0x7f91181e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.710+0000 7f911f0e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 0x7f91181f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.710+0000 7f911f0e7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c4970 con 0x7f9118157920 2024-01-20T08:30:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.710+0000 7f911f0e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c5540 con 0x7f91181506f0 2024-01-20T08:30:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.710+0000 7f911f0e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c4080 con 0x7f9118156430 2024-01-20T08:30:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.711+0000 7f911d684640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 0x7f91181f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.711+0000 7f911ce83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91181506f0 0x7f911814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.711+0000 7f911d684640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 0x7f91181f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.116:54650/0 (socket says 172.21.15.116:54650) 2024-01-20T08:30:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.711+0000 7f911d684640 1 -- 172.21.15.116:0/2213042834 learned_addr learned my addr 172.21.15.116:0/2213042834 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.714+0000 7f911d684640 1 -- 172.21.15.116:0/2213042834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9118156430 msgr2=0x7f91181e2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.714+0000 7f911d684640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9118156430 0x7f91181e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.714+0000 7f911d684640 1 -- 172.21.15.116:0/2213042834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91181506f0 msgr2=0x7f911814d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.714+0000 7f911d684640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91181506f0 0x7f911814d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.714+0000 7f911d684640 1 -- 172.21.15.116:0/2213042834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c009400 con 0x7f9118157920 2024-01-20T08:30:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.717+0000 7f911d684640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 0x7f91181f34b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f911000cbc0 tx=0x7f911000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.720+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000ad40 con 0x7f9118157920 2024-01-20T08:30:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.720+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9110012400 con 0x7f9118157920 2024-01-20T08:30:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.720+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91180c5280 con 0x7f9118157920 2024-01-20T08:30:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.720+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000aea0 con 0x7f9118157920 2024-01-20T08:30:33.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.720+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91180c4600 con 0x7f9118157920 2024-01-20T08:30:33.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.721+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911806aff0 con 0x7f9118157920 2024-01-20T08:30:33.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.722+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f9110012af0 con 0x7f9118157920 2024-01-20T08:30:33.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.821+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f91100073b0 con 0x7f9118157920 2024-01-20T08:30:33.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.821+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f9110021060 con 0x7f9118157920 2024-01-20T08:30:33.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:33.893+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "59b49896-16a3-4471-834a-3f9977004332", "id": 0} v 0) v1 -- 0x7f9118074290 con 0x7f9118157920 2024-01-20T08:30:34.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.087+0000 7f91157fa640 1 -- 172.21.15.116:0/2213042834 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "59b49896-16a3-4471-834a-3f9977004332", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9110017070 con 0x7f9118157920 2024-01-20T08:30:34.091 INFO:teuthology.orchestra.run.smithi116.stdout:0 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.088+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 msgr2=0x7f91181f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.088+0000 7f911f0e7640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 0x7f91181f34b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f911000cbc0 tx=0x7f911000a430 comp rx=0 tx=0).stop 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 shutdown_connections 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9118157920 0x7f91181f34b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9118156430 0x7f91181e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 --2- 172.21.15.116:0/2213042834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91181506f0 0x7f911814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 >> 172.21.15.116:0/2213042834 conn(0x7f9118072810 msgr2=0x7f911806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 shutdown_connections 2024-01-20T08:30:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.089+0000 7f911f0e7640 1 -- 172.21.15.116:0/2213042834 wait complete. 2024-01-20T08:30:34.102 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new bac24976-24c1-425c-988e-ace7d4640e3f 1 2024-01-20T08:30:34.115 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.113+0000 7f99e1217040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-20T08:30:34.134 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.132+0000 7efc5f64c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-20T08:30:34.169 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.166+0000 7f99e1217040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-20T08:30:34.189 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.187+0000 7efc5f64c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.199+0000 7f3fb6231640 1 Processor -- start 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.199+0000 7f3fb6231640 1 -- start start 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.200+0000 7f3fb6231640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3fb0153740 0x7f3fb00c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.200+0000 7f3fb6231640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb00c2d10 0x7f3fb006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.200+0000 7f3fb6231640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb006e350 con 0x7f3fb006f0a0 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.200+0000 7f3fb6231640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb006ddd0 con 0x7f3fb0153740 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.200+0000 7f3fb6231640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb006ffc0 con 0x7f3fb00c2d10 2024-01-20T08:30:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.201+0000 7f3faf7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3fb0153740 0x7f3fb00c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.201+0000 7f3faffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3fb006f0a0 0x7f3fb00c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49944/0 (socket says 172.21.15.116:49944) 2024-01-20T08:30:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.201+0000 7f3faffff640 1 -- 172.21.15.116:0/1602142992 learned_addr learned my addr 172.21.15.116:0/1602142992 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.201+0000 7f3faeffd640 1 --2- 172.21.15.116:0/1602142992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb00c2d10 0x7f3fb006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.204+0000 7f3fae7fc640 1 -- 172.21.15.116:0/1602142992 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009471613 0 0) 0x7f3fb006e350 con 0x7f3fb006f0a0 2024-01-20T08:30:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.204+0000 7f3fae7fc640 1 -- 172.21.15.116:0/1602142992 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f98003520 con 0x7f3fb006f0a0 2024-01-20T08:30:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.204+0000 7f3faeffd640 1 -- 172.21.15.116:0/1602142992 >> v2:172.21.15.87:3300/0 conn(0x7f3fb0153740 msgr2=0x7f3fb00c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.204+0000 7f3faeffd640 1 --2- 172.21.15.116:0/1602142992 >> v2:172.21.15.87:3300/0 conn(0x7f3fb0153740 0x7f3fb00c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.204+0000 7f3faeffd640 1 -- 172.21.15.116:0/1602142992 >> v1:172.21.15.116:6789/0 conn(0x7f3fb006f0a0 legacy=0x7f3fb00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.204+0000 7f3faeffd640 1 -- 172.21.15.116:0/1602142992 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0070280 con 0x7f3fb00c2d10 2024-01-20T08:30:34.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.206+0000 7f3faeffd640 1 --2- 172.21.15.116:0/1602142992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb00c2d10 0x7f3fb006eb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a95c9375558f4b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.212+0000 7f3fae7fc640 1 -- 172.21.15.116:0/1602142992 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0030610 con 0x7f3fb00c2d10 2024-01-20T08:30:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.212+0000 7f3fae7fc640 1 -- 172.21.15.116:0/1602142992 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0030770 con 0x7f3fb00c2d10 2024-01-20T08:30:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.212+0000 7f3fae7fc640 1 -- 172.21.15.116:0/1602142992 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0030b00 con 0x7f3fb00c2d10 2024-01-20T08:30:34.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 -- 172.21.15.116:0/1602142992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb00c2d10 msgr2=0x7f3fb006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 --2- 172.21.15.116:0/1602142992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb00c2d10 0x7f3fb006eb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002b960 comp rx=0 tx=0).stop 2024-01-20T08:30:34.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 -- 172.21.15.116:0/1602142992 shutdown_connections 2024-01-20T08:30:34.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 --2- 172.21.15.116:0/1602142992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb00c2d10 0x7f3fb006eb60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 --2- 172.21.15.116:0/1602142992 >> v2:172.21.15.87:3300/0 conn(0x7f3fb0153740 0x7f3fb00c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 -- 172.21.15.116:0/1602142992 >> 172.21.15.116:0/1602142992 conn(0x7f3fb006b6c0 msgr2=0x7f3fb006baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:34.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 -- 172.21.15.116:0/1602142992 shutdown_connections 2024-01-20T08:30:34.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.213+0000 7f3fb6231640 1 -- 172.21.15.116:0/1602142992 wait complete. 2024-01-20T08:30:34.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.214+0000 7f3fb6231640 1 Processor -- start 2024-01-20T08:30:34.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.214+0000 7f3fb6231640 1 -- start start 2024-01-20T08:30:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3fb6231640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fb006f0a0 0x7f3fb0154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3fb6231640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 0x7f3fb01e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3fb6231640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb0153740 0x7f3fb0148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3fb6231640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0148a00 con 0x7f3fb00c2d10 2024-01-20T08:30:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3fb6231640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01e52d0 con 0x7f3fb006f0a0 2024-01-20T08:30:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3fb6231640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0148440 con 0x7f3fb0153740 2024-01-20T08:30:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 0x7f3fb01e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.215+0000 7f3faf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fb006f0a0 0x7f3fb0154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.216+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 0x7f3fb01e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.116:54674/0 (socket says 172.21.15.116:54674) 2024-01-20T08:30:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.216+0000 7f3faeffd640 1 -- 172.21.15.116:0/871574663 learned_addr learned my addr 172.21.15.116:0/871574663 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.221+0000 7f3faeffd640 1 -- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb0153740 msgr2=0x7f3fb0148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.221+0000 7f3faeffd640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb0153740 0x7f3fb0148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.221+0000 7f3faeffd640 1 -- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fb006f0a0 msgr2=0x7f3fb0154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.221+0000 7f3faeffd640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fb006f0a0 0x7f3fb0154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.221+0000 7f3faeffd640 1 -- 172.21.15.116:0/871574663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0009400 con 0x7f3fb00c2d10 2024-01-20T08:30:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.221+0000 7f3faf7fe640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fb006f0a0 0x7f3fb0154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:30:34.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.223+0000 7f3faeffd640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 0x7f3fb01e5760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00098c0 tx=0x7f3fa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:34.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.228+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa003e040 con 0x7f3fb00c2d10 2024-01-20T08:30:34.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.228+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0039a40 con 0x7f3fb00c2d10 2024-01-20T08:30:34.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.228+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0039df0 con 0x7f3fb00c2d10 2024-01-20T08:30:34.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.228+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb0148720 con 0x7f3fb00c2d10 2024-01-20T08:30:34.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.228+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb01e4f60 con 0x7f3fb00c2d10 2024-01-20T08:30:34.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.229+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb01e5120 con 0x7f3fb00c2d10 2024-01-20T08:30:34.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.325+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f3fa004e7c0 con 0x7f3fb00c2d10 2024-01-20T08:30:34.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.332+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f3fa0044cf0 con 0x7f3fb00c2d10 2024-01-20T08:30:34.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.332+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f3fa0049070 con 0x7f3fb00c2d10 2024-01-20T08:30:34.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.402+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bac24976-24c1-425c-988e-ace7d4640e3f", "id": 1} v 0) v1 -- 0x7f3fb01e4ae0 con 0x7f3fb00c2d10 2024-01-20T08:30:34.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.445+0000 7f3f97fff640 1 -- 172.21.15.116:0/871574663 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bac24976-24c1-425c-988e-ace7d4640e3f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3fb01e4ae0 con 0x7f3fb00c2d10 2024-01-20T08:30:34.448 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 msgr2=0x7f3fb01e5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 0x7f3fb01e5760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00098c0 tx=0x7f3fa0004730 comp rx=0 tx=0).stop 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 shutdown_connections 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fb0153740 0x7f3fb0148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fb00c2d10 0x7f3fb01e5760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 --2- 172.21.15.116:0/871574663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fb006f0a0 0x7f3fb0154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 >> 172.21.15.116:0/871574663 conn(0x7f3fb006b6c0 msgr2=0x7f3fb0151fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 shutdown_connections 2024-01-20T08:30:34.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.446+0000 7f3fb6231640 1 -- 172.21.15.116:0/871574663 wait complete. 2024-01-20T08:30:34.460 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 28b3c7c6-3c5e-4309-a825-0945264488ae 2 2024-01-20T08:30:34.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.612+0000 7fe4c981e640 1 Processor -- start 2024-01-20T08:30:34.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.612+0000 7fe4c981e640 1 -- start start 2024-01-20T08:30:34.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.613+0000 7fe4c981e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c40c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.613+0000 7fe4c981e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c4153a40 con 0x7fe4c4153660 2024-01-20T08:30:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.613+0000 7fe4c981e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c40c3a60 con 0x7fe4c40c2d80 2024-01-20T08:30:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.613+0000 7fe4c981e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c406d420 con 0x7fe4c415a1b0 2024-01-20T08:30:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.614+0000 7fe4c27fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c40c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.614+0000 7fe4c37fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe4c40c2d80 0x7fe4c40ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49954/0 (socket says 172.21.15.116:49954) 2024-01-20T08:30:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.614+0000 7fe4c37fe640 1 -- 172.21.15.116:0/3499542177 learned_addr learned my addr 172.21.15.116:0/3499542177 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.616+0000 7fe4c27fc640 1 -- 172.21.15.116:0/3499542177 >> v1:172.21.15.116:6789/0 conn(0x7fe4c40c2d80 legacy=0x7fe4c40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.616+0000 7fe4c27fc640 1 -- 172.21.15.116:0/3499542177 >> v1:172.21.15.87:6789/0 conn(0x7fe4c4153660 legacy=0x7fe4c40c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.616+0000 7fe4c1ffb640 1 -- 172.21.15.116:0/3499542177 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748460302 0 0) 0x7fe4c40c3a60 con 0x7fe4c40c2d80 2024-01-20T08:30:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.616+0000 7fe4c27fc640 1 -- 172.21.15.116:0/3499542177 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c40c95d0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.618+0000 7fe4c27fc640 1 --2- 172.21.15.116:0/3499542177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c40c97c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009dc0 tx=0x7fe4b802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645d9ce3ea597374 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:34.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.622+0000 7fe4c1ffb640 1 -- 172.21.15.116:0/3499542177 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b802ccd0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.622+0000 7fe4c1ffb640 1 -- 172.21.15.116:0/3499542177 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b802ce30 con 0x7fe4c415a1b0 2024-01-20T08:30:34.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.622+0000 7fe4c1ffb640 1 -- 172.21.15.116:0/3499542177 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8034660 con 0x7fe4c415a1b0 2024-01-20T08:30:34.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 -- 172.21.15.116:0/3499542177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 msgr2=0x7fe4c40c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 --2- 172.21.15.116:0/3499542177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c40c97c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009dc0 tx=0x7fe4b802bf20 comp rx=0 tx=0).stop 2024-01-20T08:30:34.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 -- 172.21.15.116:0/3499542177 shutdown_connections 2024-01-20T08:30:34.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 --2- 172.21.15.116:0/3499542177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c40c97c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 -- 172.21.15.116:0/3499542177 >> 172.21.15.116:0/3499542177 conn(0x7fe4c406a5b0 msgr2=0x7fe4c406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:34.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 -- 172.21.15.116:0/3499542177 shutdown_connections 2024-01-20T08:30:34.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.623+0000 7fe4c981e640 1 -- 172.21.15.116:0/3499542177 wait complete. 2024-01-20T08:30:34.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.624+0000 7fe4c981e640 1 Processor -- start 2024-01-20T08:30:34.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.624+0000 7fe4c981e640 1 -- start start 2024-01-20T08:30:34.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.625+0000 7fe4c981e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4c40c2d80 0x7fe4c4146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.625+0000 7fe4c981e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4c4153660 0x7fe4c414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.625+0000 7fe4c981e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.625+0000 7fe4c981e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c414c500 con 0x7fe4c4153660 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.625+0000 7fe4c981e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4146a80 con 0x7fe4c40c2d80 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.625+0000 7fe4c981e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4148180 con 0x7fe4c415a1b0 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.626+0000 7fe4c37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.626+0000 7fe4c27fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4c4153660 0x7fe4c414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.626+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4c40c2d80 0x7fe4c4146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.626+0000 7fe4c37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c414cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:35202/0 (socket says 172.21.15.116:35202) 2024-01-20T08:30:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.626+0000 7fe4c37fe640 1 -- 172.21.15.116:0/2518839656 learned_addr learned my addr 172.21.15.116:0/2518839656 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.629+0000 7fe4c37fe640 1 -- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4c40c2d80 msgr2=0x7fe4c4146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.629+0000 7fe4c37fe640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4c40c2d80 0x7fe4c4146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.629+0000 7fe4c37fe640 1 -- 172.21.15.116:0/2518839656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4c4153660 msgr2=0x7fe4c414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.629+0000 7fe4c37fe640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4c4153660 0x7fe4c414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.629+0000 7fe4c37fe640 1 -- 172.21.15.116:0/2518839656 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8009a70 con 0x7fe4c415a1b0 2024-01-20T08:30:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.629+0000 7fe4c2ffd640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4c40c2d80 0x7fe4c4146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:30:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.632+0000 7fe4c37fe640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c414cc30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c440 tx=0x7fe4b400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.635+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40074e0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.635+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4007640 con 0x7fe4c415a1b0 2024-01-20T08:30:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.635+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4004a20 con 0x7fe4c415a1b0 2024-01-20T08:30:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.635+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c41483f0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.635+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c41489b0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.636+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c4148ba0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.727 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.724+0000 7f99e1217040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-20T08:30:34.730 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.728+0000 7efc5f64c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-20T08:30:34.779 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.776+0000 7f99e1217040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-20T08:30:34.782 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.780+0000 7efc5f64c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-20T08:30:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.806+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fe4b401d3f0 con 0x7fe4c415a1b0 2024-01-20T08:30:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.806+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fe4b4013a30 con 0x7fe4c415a1b0 2024-01-20T08:30:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.806+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fe4b4013c40 con 0x7fe4c415a1b0 2024-01-20T08:30:34.823 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.820+0000 7f99e1217040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-20T08:30:34.828 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.825+0000 7efc5f64c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-20T08:30:34.876 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.873+0000 7f99e1217040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-20T08:30:34.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:34.876+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "28b3c7c6-3c5e-4309-a825-0945264488ae", "id": 2} v 0) v1 -- 0x7fe4c4074290 con 0x7fe4c415a1b0 2024-01-20T08:30:34.881 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.878+0000 7efc5f64c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-20T08:30:34.917 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:34.915+0000 7f99e1217040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-20T08:30:34.923 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:34.920+0000 7efc5f64c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-20T08:30:35.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.006+0000 7fe4abfff640 1 -- 172.21.15.116:0/2518839656 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "28b3c7c6-3c5e-4309-a825-0945264488ae", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe4b40135d0 con 0x7fe4c415a1b0 2024-01-20T08:30:35.009 INFO:teuthology.orchestra.run.smithi116.stdout:2 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 msgr2=0x7fe4c414cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c414cc30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c440 tx=0x7fe4b400c900 comp rx=0 tx=0).stop 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 shutdown_connections 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4c415a1b0 0x7fe4c414cc30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4c4153660 0x7fe4c414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 --2- 172.21.15.116:0/2518839656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4c40c2d80 0x7fe4c4146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 >> 172.21.15.116:0/2518839656 conn(0x7fe4c406a5b0 msgr2=0x7fe4c40c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:35.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 shutdown_connections 2024-01-20T08:30:35.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.007+0000 7fe4c981e640 1 -- 172.21.15.116:0/2518839656 wait complete. 2024-01-20T08:30:35.020 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 3a06fe92-a35d-439c-a9be-ed5afb82c6ed 3 2024-01-20T08:30:35.024 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.021+0000 7f99e1217040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-20T08:30:35.028 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.026+0000 7efc5f64c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-20T08:30:35.093 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.090+0000 7f99e1217040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-20T08:30:35.101 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.098+0000 7efc5f64c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-20T08:30:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.117+0000 7f8e04dd0640 1 Processor -- start 2024-01-20T08:30:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.117+0000 7f8e04dd0640 1 -- start start 2024-01-20T08:30:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.118+0000 7f8e04dd0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e001506f0 0x7f8e0015a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.118+0000 7f8e04dd0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8e00156430 0x7f8e0006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.118+0000 7f8e04dd0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e000c0f20 con 0x7f8e00157920 2024-01-20T08:30:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.118+0000 7f8e04dd0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e000c91c0 con 0x7f8e00156430 2024-01-20T08:30:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.118+0000 7f8e04dd0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0015a990 con 0x7f8e001506f0 2024-01-20T08:30:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.119+0000 7f8dfe59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e001506f0 0x7f8e0015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.119+0000 7f8dfed9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8e00156430 0x7f8e0006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.119+0000 7f8dfdd9b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8e00157920 0x7f8e000c1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49968/0 (socket says 172.21.15.116:49968) 2024-01-20T08:30:35.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.119+0000 7f8dfdd9b640 1 -- 172.21.15.116:0/3278330192 learned_addr learned my addr 172.21.15.116:0/3278330192 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:35.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.122+0000 7f8dfd59a640 1 -- 172.21.15.116:0/3278330192 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978555226 0 0) 0x7f8e000c0f20 con 0x7f8e00157920 2024-01-20T08:30:35.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.122+0000 7f8dfd59a640 1 -- 172.21.15.116:0/3278330192 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de8003520 con 0x7f8e00157920 2024-01-20T08:30:35.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.123+0000 7f8dfe59c640 1 -- 172.21.15.116:0/3278330192 >> v2:172.21.15.87:3300/0 conn(0x7f8e00156430 msgr2=0x7f8e0006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:35.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.123+0000 7f8dfe59c640 1 --2- 172.21.15.116:0/3278330192 >> v2:172.21.15.87:3300/0 conn(0x7f8e00156430 0x7f8e0006fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.123+0000 7f8dfe59c640 1 -- 172.21.15.116:0/3278330192 >> v1:172.21.15.116:6789/0 conn(0x7f8e00157920 legacy=0x7f8e000c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.123+0000 7f8dfe59c640 1 -- 172.21.15.116:0/3278330192 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00156da0 con 0x7f8e001506f0 2024-01-20T08:30:35.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.125+0000 7f8dfe59c640 1 --2- 172.21.15.116:0/3278330192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e001506f0 0x7f8e0015a450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097a0 tx=0x7f8df002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5821a7cab51e7a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:35.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.130+0000 7f8dfd59a640 1 -- 172.21.15.116:0/3278330192 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df002e070 con 0x7f8e001506f0 2024-01-20T08:30:35.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.130+0000 7f8dfd59a640 1 -- 172.21.15.116:0/3278330192 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00315a0 con 0x7f8e001506f0 2024-01-20T08:30:35.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.130+0000 7f8dfd59a640 1 -- 172.21.15.116:0/3278330192 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0031930 con 0x7f8e001506f0 2024-01-20T08:30:35.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 -- 172.21.15.116:0/3278330192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e001506f0 msgr2=0x7f8e0015a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/3278330192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e001506f0 0x7f8e0015a450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097a0 tx=0x7f8df002b840 comp rx=0 tx=0).stop 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 -- 172.21.15.116:0/3278330192 shutdown_connections 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/3278330192 >> v2:172.21.15.87:3300/0 conn(0x7f8e00156430 0x7f8e0006fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/3278330192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e001506f0 0x7f8e0015a450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 -- 172.21.15.116:0/3278330192 >> 172.21.15.116:0/3278330192 conn(0x7f8e00072810 msgr2=0x7f8e0006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 -- 172.21.15.116:0/3278330192 shutdown_connections 2024-01-20T08:30:35.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.131+0000 7f8e04dd0640 1 -- 172.21.15.116:0/3278330192 wait complete. 2024-01-20T08:30:35.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.133+0000 7f8e04dd0640 1 Processor -- start 2024-01-20T08:30:35.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.133+0000 7f8e04dd0640 1 -- start start 2024-01-20T08:30:35.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8e04dd0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e001506f0 0x7f8e001ec850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8e04dd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e00156430 0x7f8e00145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8e04dd0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 0x7f8e00158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8e04dd0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0014e400 con 0x7f8e001506f0 2024-01-20T08:30:35.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8e04dd0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0014b0d0 con 0x7f8e00156430 2024-01-20T08:30:35.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8e04dd0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0014a5f0 con 0x7f8e00157920 2024-01-20T08:30:35.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8dfe59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e001506f0 0x7f8e001ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8dfed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 0x7f8e00158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.134+0000 7f8dfdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e00156430 0x7f8e00145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.135+0000 7f8dfed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 0x7f8e00158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:35218/0 (socket says 172.21.15.116:35218) 2024-01-20T08:30:35.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.135+0000 7f8dfed9d640 1 -- 172.21.15.116:0/4076310487 learned_addr learned my addr 172.21.15.116:0/4076310487 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:35.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.138+0000 7f8dfed9d640 1 -- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e00156430 msgr2=0x7f8e00145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.138+0000 7f8dfed9d640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e00156430 0x7f8e00145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.138+0000 7f8dfed9d640 1 -- 172.21.15.116:0/4076310487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e001506f0 msgr2=0x7f8e001ec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.138+0000 7f8dfed9d640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e001506f0 0x7f8e001ec850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.138+0000 7f8dfed9d640 1 -- 172.21.15.116:0/4076310487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0009400 con 0x7f8e00157920 2024-01-20T08:30:35.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.140+0000 7f8dfed9d640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 0x7f8e00158d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8df800efc0 tx=0x7f8df800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:35.145 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.142+0000 7f99e1217040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-20T08:30:35.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.144+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8014480 con 0x7f8e00157920 2024-01-20T08:30:35.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.144+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800f040 con 0x7f8e00157920 2024-01-20T08:30:35.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.144+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e001ecd90 con 0x7f8e00157920 2024-01-20T08:30:35.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.144+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8014820 con 0x7f8e00157920 2024-01-20T08:30:35.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.144+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e001ebc30 con 0x7f8e00157920 2024-01-20T08:30:35.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.145+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00156160 con 0x7f8e00157920 2024-01-20T08:30:35.154 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.151+0000 7efc5f64c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-20T08:30:35.197 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.194+0000 7f99e1217040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-20T08:30:35.207 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.204+0000 7efc5f64c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-20T08:30:35.243 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.240+0000 7f99e1217040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-20T08:30:35.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.247+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f8df8014d00 con 0x7f8e00157920 2024-01-20T08:30:35.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.247+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f8df8013870 con 0x7f8e00157920 2024-01-20T08:30:35.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.247+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f8df803bcf0 con 0x7f8e00157920 2024-01-20T08:30:35.257 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.254+0000 7efc5f64c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-20T08:30:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.318+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3a06fe92-a35d-439c-a9be-ed5afb82c6ed", "id": 3} v 0) v1 -- 0x7f8e00074290 con 0x7f8e00157920 2024-01-20T08:30:35.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.399+0000 7f8deeffd640 1 -- 172.21.15.116:0/4076310487 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a06fe92-a35d-439c-a9be-ed5afb82c6ed", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8df802a030 con 0x7f8e00157920 2024-01-20T08:30:35.402 INFO:teuthology.orchestra.run.smithi116.stdout:3 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.399+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 msgr2=0x7f8e00158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.399+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 0x7f8e00158d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8df800efc0 tx=0x7f8df800c490 comp rx=0 tx=0).stop 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 shutdown_connections 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e00157920 0x7f8e00158d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e00156430 0x7f8e00145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 --2- 172.21.15.116:0/4076310487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e001506f0 0x7f8e001ec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 >> 172.21.15.116:0/4076310487 conn(0x7f8e00072810 msgr2=0x7f8e00157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 shutdown_connections 2024-01-20T08:30:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.400+0000 7f8e04dd0640 1 -- 172.21.15.116:0/4076310487 wait complete. 2024-01-20T08:30:35.413 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new d6386370-b971-4716-a270-0192527c4863 4 2024-01-20T08:30:35.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.554+0000 7f5f9270d640 1 Processor -- start 2024-01-20T08:30:35.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.554+0000 7f5f9270d640 1 -- start start 2024-01-20T08:30:35.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.555+0000 7f5f9270d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c1506f0 0x7f5f8c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.555+0000 7f5f9270d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5f8c157920 0x7f5f8c0c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.555+0000 7f5f9270d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f8c156430 0x7f5f8c06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.555+0000 7f5f9270d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0c91c0 con 0x7f5f8c156430 2024-01-20T08:30:35.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.555+0000 7f5f9270d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c15a990 con 0x7f5f8c157920 2024-01-20T08:30:35.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.555+0000 7f5f9270d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0c0f20 con 0x7f5f8c1506f0 2024-01-20T08:30:35.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.558+0000 7f5f8b7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5f8c157920 0x7f5f8c0c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.560+0000 7f5f8b7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5f8c157920 0x7f5f8c0c1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59824/0 (socket says 172.21.15.116:59824) 2024-01-20T08:30:35.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.560+0000 7f5f8b7fe640 1 -- 172.21.15.116:0/912362492 learned_addr learned my addr 172.21.15.116:0/912362492 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:35.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.561+0000 7f5f8bfff640 1 --2- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c1506f0 0x7f5f8c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.564+0000 7f5f8b7fe640 1 -- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c1506f0 msgr2=0x7f5f8c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:35.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.564+0000 7f5f8b7fe640 1 --2- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c1506f0 0x7f5f8c15a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.564+0000 7f5f8b7fe640 1 -- 172.21.15.116:0/912362492 >> v2:172.21.15.87:3300/0 conn(0x7f5f8c156430 msgr2=0x7f5f8c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:35.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.564+0000 7f5f8b7fe640 1 --2- 172.21.15.116:0/912362492 >> v2:172.21.15.87:3300/0 conn(0x7f5f8c156430 0x7f5f8c06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.564+0000 7f5f8b7fe640 1 -- 172.21.15.116:0/912362492 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c156da0 con 0x7f5f8c157920 2024-01-20T08:30:35.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.567+0000 7f5f8b7fe640 1 --2- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c157920 0x7f5f8c0c1990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c156f90 tx=0x7f5f8002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee9cea85457c0a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:35.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.626+0000 7f5f8affd640 1 -- 172.21.15.116:0/912362492 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80031640 con 0x7f5f8c157920 2024-01-20T08:30:35.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.626+0000 7f5f8affd640 1 -- 172.21.15.116:0/912362492 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f800317a0 con 0x7f5f8c157920 2024-01-20T08:30:35.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.626+0000 7f5f8affd640 1 -- 172.21.15.116:0/912362492 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80031b30 con 0x7f5f8c157920 2024-01-20T08:30:35.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 -- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c157920 msgr2=0x7f5f8c0c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 --2- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c157920 0x7f5f8c0c1990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c156f90 tx=0x7f5f8002b8f0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 -- 172.21.15.116:0/912362492 shutdown_connections 2024-01-20T08:30:35.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 --2- 172.21.15.116:0/912362492 >> v2:172.21.15.87:3300/0 conn(0x7f5f8c156430 0x7f5f8c06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 --2- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c157920 0x7f5f8c0c1990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 --2- 172.21.15.116:0/912362492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c1506f0 0x7f5f8c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 -- 172.21.15.116:0/912362492 >> 172.21.15.116:0/912362492 conn(0x7f5f8c072810 msgr2=0x7f5f8c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:35.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 -- 172.21.15.116:0/912362492 shutdown_connections 2024-01-20T08:30:35.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.627+0000 7f5f9270d640 1 -- 172.21.15.116:0/912362492 wait complete. 2024-01-20T08:30:35.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.628+0000 7f5f9270d640 1 Processor -- start 2024-01-20T08:30:35.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.628+0000 7f5f9270d640 1 -- start start 2024-01-20T08:30:35.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.629+0000 7f5f9270d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f8c1506f0 0x7f5f8c14a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.629+0000 7f5f9270d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 0x7f5f8c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.629+0000 7f5f9270d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c157920 0x7f5f8c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.629+0000 7f5f9270d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c14ae70 con 0x7f5f8c1506f0 2024-01-20T08:30:35.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.629+0000 7f5f9270d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c149320 con 0x7f5f8c156430 2024-01-20T08:30:35.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.629+0000 7f5f9270d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c149600 con 0x7f5f8c157920 2024-01-20T08:30:35.633 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.630+0000 7f99e1217040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-20T08:30:35.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.630+0000 7f5f8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f8c1506f0 0x7f5f8c14a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.630+0000 7f5f8b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 0x7f5f8c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.630+0000 7f5f90caa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c157920 0x7f5f8c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.630+0000 7f5f8b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 0x7f5f8c0caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59826/0 (socket says 172.21.15.116:59826) 2024-01-20T08:30:35.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.630+0000 7f5f8b7fe640 1 -- 172.21.15.116:0/461951185 learned_addr learned my addr 172.21.15.116:0/461951185 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:35.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f90caa640 1 -- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 msgr2=0x7f5f8c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f90caa640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 0x7f5f8c0caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f90caa640 1 -- 172.21.15.116:0/461951185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f8c1506f0 msgr2=0x7f5f8c14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f90caa640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f8c1506f0 0x7f5f8c14a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f90caa640 1 -- 172.21.15.116:0/461951185 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f800094d0 con 0x7f5f8c157920 2024-01-20T08:30:35.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f8b7fe640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 0x7f5f8c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:30:35.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.633+0000 7f5f8bfff640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f8c1506f0 0x7f5f8c14a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:30:35.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.635+0000 7f5f90caa640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c157920 0x7f5f8c14d290 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400be20 tx=0x7f5f8400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:35.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.639+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400ce70 con 0x7f5f8c157920 2024-01-20T08:30:35.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.639+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84014440 con 0x7f5f8c157920 2024-01-20T08:30:35.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.639+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c14b0c0 con 0x7f5f8c157920 2024-01-20T08:30:35.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.639+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84014810 con 0x7f5f8c157920 2024-01-20T08:30:35.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.639+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c148570 con 0x7f5f8c157920 2024-01-20T08:30:35.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.640+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c0c8340 con 0x7f5f8c157920 2024-01-20T08:30:35.658 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.655+0000 7efc5f64c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-20T08:30:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.738+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f5f84014cf0 con 0x7f5f8c157920 2024-01-20T08:30:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.738+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f5f8401ca80 con 0x7f5f8c157920 2024-01-20T08:30:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.738+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f5f84019070 con 0x7f5f8c157920 2024-01-20T08:30:35.766 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.763+0000 7f99e1217040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-20T08:30:35.794 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.791+0000 7efc5f64c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-20T08:30:35.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.809+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d6386370-b971-4716-a270-0192527c4863", "id": 4} v 0) v1 -- 0x7f5f8c074290 con 0x7f5f8c157920 2024-01-20T08:30:35.875 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.873+0000 7f99e1217040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-20T08:30:35.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.881+0000 7f5f88ff9640 1 -- 172.21.15.116:0/461951185 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6386370-b971-4716-a270-0192527c4863", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f8401c530 con 0x7f5f8c157920 2024-01-20T08:30:35.884 INFO:teuthology.orchestra.run.smithi116.stdout:4 2024-01-20T08:30:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c157920 msgr2=0x7f5f8c14d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c157920 0x7f5f8c14d290 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400be20 tx=0x7f5f8400bf10 comp rx=0 tx=0).stop 2024-01-20T08:30:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 shutdown_connections 2024-01-20T08:30:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c157920 0x7f5f8c14d290 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c156430 0x7f5f8c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 --2- 172.21.15.116:0/461951185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f8c1506f0 0x7f5f8c14a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 >> 172.21.15.116:0/461951185 conn(0x7f5f8c072810 msgr2=0x7f5f8c06e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 shutdown_connections 2024-01-20T08:30:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.882+0000 7f5f9270d640 1 -- 172.21.15.116:0/461951185 wait complete. 2024-01-20T08:30:35.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new c78a9699-c969-4c6a-8cc6-3159de537dac 5 2024-01-20T08:30:35.914 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.912+0000 7efc5f64c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-20T08:30:35.920 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:35.917+0000 7f99e1217040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-20T08:30:35.962 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:35.959+0000 7efc5f64c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-20T08:30:35.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.990+0000 7fe4e6010640 1 Processor -- start 2024-01-20T08:30:35.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.991+0000 7fe4e6010640 1 -- start start 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.991+0000 7fe4e6010640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe4e00719b0 0x7fe4e00724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.991+0000 7fe4e6010640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e00c6720 0x7fe4e0159890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.991+0000 7fe4e6010640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e00c5f20 con 0x7fe4e0071d90 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.991+0000 7fe4e6010640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e006b7f0 con 0x7fe4e00719b0 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.991+0000 7fe4e6010640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c5840 con 0x7fe4e00c6720 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.992+0000 7fe4df7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe4e00719b0 0x7fe4e00724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.992+0000 7fe4dffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e00c6720 0x7fe4e0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:35.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.992+0000 7fe4deffd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe4e0071d90 0x7fe4e0153720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.116:36346/0 (socket says 172.21.15.116:36346) 2024-01-20T08:30:35.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.992+0000 7fe4deffd640 1 -- 172.21.15.116:0/3195190275 learned_addr learned my addr 172.21.15.116:0/3195190275 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:35.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.995+0000 7fe4df7fe640 1 -- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e00c6720 msgr2=0x7fe4e0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:35.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.995+0000 7fe4df7fe640 1 --2- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e00c6720 0x7fe4e0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:35.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.995+0000 7fe4de7fc640 1 -- 172.21.15.116:0/3195190275 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724549943 0 0) 0x7fe4e00c5f20 con 0x7fe4e0071d90 2024-01-20T08:30:35.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.995+0000 7fe4df7fe640 1 -- 172.21.15.116:0/3195190275 >> v1:172.21.15.87:6789/0 conn(0x7fe4e0071d90 legacy=0x7fe4e0153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:35.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.995+0000 7fe4df7fe640 1 -- 172.21.15.116:0/3195190275 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00c6170 con 0x7fe4e00719b0 2024-01-20T08:30:35.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:35.996+0000 7fe4df7fe640 1 --2- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00719b0 0x7fe4e00724c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40097a0 tx=0x7fe4d402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b599ba5028d2cdf server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:36.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.000+0000 7fe4de7fc640 1 -- 172.21.15.116:0/3195190275 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4030610 con 0x7fe4e00719b0 2024-01-20T08:30:36.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.000+0000 7fe4de7fc640 1 -- 172.21.15.116:0/3195190275 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4030770 con 0x7fe4e00719b0 2024-01-20T08:30:36.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.000+0000 7fe4de7fc640 1 -- 172.21.15.116:0/3195190275 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4030b00 con 0x7fe4e00719b0 2024-01-20T08:30:36.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 -- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00719b0 msgr2=0x7fe4e00724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 --2- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00719b0 0x7fe4e00724c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40097a0 tx=0x7fe4d402b960 comp rx=0 tx=0).stop 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 -- 172.21.15.116:0/3195190275 shutdown_connections 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 --2- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e00c6720 0x7fe4e0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 --2- 172.21.15.116:0/3195190275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00719b0 0x7fe4e00724c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 -- 172.21.15.116:0/3195190275 >> 172.21.15.116:0/3195190275 conn(0x7fe4e006f9a0 msgr2=0x7fe4e00c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 -- 172.21.15.116:0/3195190275 shutdown_connections 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.001+0000 7fe4e6010640 1 -- 172.21.15.116:0/3195190275 wait complete. 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.002+0000 7fe4e6010640 1 Processor -- start 2024-01-20T08:30:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 -- start start 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4e00719b0 0x7fe4e00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e0071d90 0x7fe4e01e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 0x7fe4e01e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01e4c50 con 0x7fe4e00719b0 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01e4f30 con 0x7fe4e00c6720 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.003+0000 7fe4e6010640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01e5210 con 0x7fe4e0071d90 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.004+0000 7fe4dffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 0x7fe4e01e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.004+0000 7fe4df7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4e00719b0 0x7fe4e00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.004+0000 7fe4deffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e0071d90 0x7fe4e01e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.004+0000 7fe4dffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 0x7fe4e01e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59848/0 (socket says 172.21.15.116:59848) 2024-01-20T08:30:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.004+0000 7fe4dffff640 1 -- 172.21.15.116:0/1989803738 learned_addr learned my addr 172.21.15.116:0/1989803738 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.008+0000 7fe4dffff640 1 -- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e0071d90 msgr2=0x7fe4e01e5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.008+0000 7fe4dffff640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e0071d90 0x7fe4e01e5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.008+0000 7fe4dffff640 1 -- 172.21.15.116:0/1989803738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4e00719b0 msgr2=0x7fe4e00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.008+0000 7fe4dffff640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4e00719b0 0x7fe4e00c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.008+0000 7fe4dffff640 1 -- 172.21.15.116:0/1989803738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009400 con 0x7fe4e00c6720 2024-01-20T08:30:36.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.010+0000 7fe4dffff640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 0x7fe4e01e6230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800efc0 tx=0x7fe4d800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:36.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.013+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8009ba0 con 0x7fe4e00c6720 2024-01-20T08:30:36.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.013+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d800f040 con 0x7fe4e00c6720 2024-01-20T08:30:36.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.013+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80146c0 con 0x7fe4e00c6720 2024-01-20T08:30:36.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.013+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e01e54f0 con 0x7fe4e00c6720 2024-01-20T08:30:36.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.013+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e014fa20 con 0x7fe4e00c6720 2024-01-20T08:30:36.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.014+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e01e5a10 con 0x7fe4e00c6720 2024-01-20T08:30:36.023 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:36.020+0000 7f99e1217040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-20T08:30:36.090 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:36.087+0000 7efc5f64c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-20T08:30:36.117 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:36.114+0000 7f99e1217040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-20T08:30:36.184 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:36.182+0000 7efc5f64c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-20T08:30:36.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.226+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fe4d8014ba0 con 0x7fe4e00c6720 2024-01-20T08:30:36.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.226+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe4d8013960 con 0x7fe4e00c6720 2024-01-20T08:30:36.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.226+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fe4e01e5a10 con 0x7fe4e00c6720 2024-01-20T08:30:36.256 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:36.254+0000 7f99e1217040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-20T08:30:36.298 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:36.296+0000 7f99e1217040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-20T08:30:36.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.296+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c78a9699-c969-4c6a-8cc6-3159de537dac", "id": 5} v 0) v1 -- 0x7fe4e0074290 con 0x7fe4e00c6720 2024-01-20T08:30:36.325 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:36.322+0000 7efc5f64c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-20T08:30:36.369 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:36.367+0000 7efc5f64c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-20T08:30:36.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.407+0000 7fe4c7fff640 1 -- 172.21.15.116:0/1989803738 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c78a9699-c969-4c6a-8cc6-3159de537dac", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe4d8018020 con 0x7fe4e00c6720 2024-01-20T08:30:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:5 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 msgr2=0x7fe4e01e6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 0x7fe4e01e6230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800efc0 tx=0x7fe4d800c490 comp rx=0 tx=0).stop 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 shutdown_connections 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4e00c6720 0x7fe4e01e6230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4e0071d90 0x7fe4e01e5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 --2- 172.21.15.116:0/1989803738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4e00719b0 0x7fe4e00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 >> 172.21.15.116:0/1989803738 conn(0x7fe4e006f9a0 msgr2=0x7fe4e0151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 shutdown_connections 2024-01-20T08:30:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.408+0000 7fe4e6010640 1 -- 172.21.15.116:0/1989803738 wait complete. 2024-01-20T08:30:36.421 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 389dcf96-e639-45ee-b53c-0efd01b1be3a 6 2024-01-20T08:30:36.446 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:36.443+0000 7f99e1217040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-20T08:30:36.521 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:36.519+0000 7efc5f64c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-20T08:30:36.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.565+0000 7f092ff05640 1 Processor -- start 2024-01-20T08:30:36.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.565+0000 7f092ff05640 1 -- start start 2024-01-20T08:30:36.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.566+0000 7f092ff05640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09280c58f0 0x7f09280cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.566+0000 7f092ff05640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09280b1fb0 con 0x7f09280c5510 2024-01-20T08:30:36.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.566+0000 7f092ff05640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09281574b0 con 0x7f092815b6d0 2024-01-20T08:30:36.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.566+0000 7f092ff05640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0928157750 con 0x7f09280c58f0 2024-01-20T08:30:36.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.568+0000 7f092e4a2640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f092815b6d0 0x7f09281403f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49982/0 (socket says 172.21.15.116:49982) 2024-01-20T08:30:36.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.568+0000 7f092e4a2640 1 -- 172.21.15.116:0/793872874 learned_addr learned my addr 172.21.15.116:0/793872874 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.582+0000 7f092d4a0640 1 --2- 172.21.15.116:0/793872874 >> v2:172.21.15.87:3300/0 conn(0x7f09280c58f0 0x7f09280cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.595+0000 7f092d4a0640 1 -- 172.21.15.116:0/793872874 >> v1:172.21.15.116:6789/0 conn(0x7f092815b6d0 legacy=0x7f09281403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.595+0000 7f092d4a0640 1 -- 172.21.15.116:0/793872874 >> v1:172.21.15.87:6789/0 conn(0x7f09280c5510 legacy=0x7f092815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.595+0000 7f092d4a0640 1 -- 172.21.15.116:0/793872874 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928157a30 con 0x7f09280c58f0 2024-01-20T08:30:36.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.602+0000 7f092d4a0640 1 --2- 172.21.15.116:0/793872874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09280c58f0 0x7f09280cc7b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09180097d0 tx=0x7f091802b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374852d4fd4bac93 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:36.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.607+0000 7f092cc9f640 1 -- 172.21.15.116:0/793872874 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918030890 con 0x7f09280c58f0 2024-01-20T08:30:36.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.607+0000 7f092cc9f640 1 -- 172.21.15.116:0/793872874 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180309f0 con 0x7f09280c58f0 2024-01-20T08:30:36.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.607+0000 7f092cc9f640 1 -- 172.21.15.116:0/793872874 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918030d80 con 0x7f09280c58f0 2024-01-20T08:30:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 -- 172.21.15.116:0/793872874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09280c58f0 msgr2=0x7f09280cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 --2- 172.21.15.116:0/793872874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09280c58f0 0x7f09280cc7b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09180097d0 tx=0x7f091802b9d0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 -- 172.21.15.116:0/793872874 shutdown_connections 2024-01-20T08:30:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 --2- 172.21.15.116:0/793872874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09280c58f0 0x7f09280cc7b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 -- 172.21.15.116:0/793872874 >> 172.21.15.116:0/793872874 conn(0x7f0928072810 msgr2=0x7f092806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 -- 172.21.15.116:0/793872874 shutdown_connections 2024-01-20T08:30:36.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.608+0000 7f092ff05640 1 -- 172.21.15.116:0/793872874 wait complete. 2024-01-20T08:30:36.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.610+0000 7f092ff05640 1 Processor -- start 2024-01-20T08:30:36.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.610+0000 7f092ff05640 1 -- start start 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.611+0000 7f092ff05640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09280c5510 0x7f09281f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.611+0000 7f092ff05640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09280c58f0 0x7f092806a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.611+0000 7f092ff05640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 0x7f0928158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.611+0000 7f092ff05640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092806b1f0 con 0x7f092815b6d0 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.611+0000 7f092ff05640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09281f1d40 con 0x7f09280c58f0 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.611+0000 7f092ff05640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280cb8d0 con 0x7f09280c5510 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.612+0000 7f092e4a2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 0x7f0928158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.612+0000 7f092dca1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09280c5510 0x7f09281f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.612+0000 7f092d4a0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09280c58f0 0x7f092806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.612+0000 7f092e4a2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 0x7f0928158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.116:54732/0 (socket says 172.21.15.116:54732) 2024-01-20T08:30:36.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.612+0000 7f092e4a2640 1 -- 172.21.15.116:0/2592088110 learned_addr learned my addr 172.21.15.116:0/2592088110 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:36.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.615+0000 7f092e4a2640 1 -- 172.21.15.116:0/2592088110 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09280c5510 msgr2=0x7f09281f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.615+0000 7f092e4a2640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09280c5510 0x7f09281f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.615+0000 7f092e4a2640 1 -- 172.21.15.116:0/2592088110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09280c58f0 msgr2=0x7f092806a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.615+0000 7f092e4a2640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09280c58f0 0x7f092806a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.615+0000 7f092e4a2640 1 -- 172.21.15.116:0/2592088110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918009400 con 0x7f092815b6d0 2024-01-20T08:30:36.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.618+0000 7f092e4a2640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 0x7f0928158670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f092000ca80 tx=0x7f092000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:36.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.621+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000abd0 con 0x7f092815b6d0 2024-01-20T08:30:36.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.621+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092000ad30 con 0x7f092815b6d0 2024-01-20T08:30:36.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.621+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09281ee7e0 con 0x7f092815b6d0 2024-01-20T08:30:36.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.621+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920012600 con 0x7f092815b6d0 2024-01-20T08:30:36.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.621+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09281eda20 con 0x7f092815b6d0 2024-01-20T08:30:36.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.622+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09280c8dd0 con 0x7f092815b6d0 2024-01-20T08:30:36.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.717+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f0920012ae0 con 0x7f092815b6d0 2024-01-20T08:30:36.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.721+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f0920007660 con 0x7f092815b6d0 2024-01-20T08:30:36.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.721+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f0920021060 con 0x7f092815b6d0 2024-01-20T08:30:36.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.792+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "389dcf96-e639-45ee-b53c-0efd01b1be3a", "id": 6} v 0) v1 -- 0x7f092806e7c0 con 0x7f092815b6d0 2024-01-20T08:30:36.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.819+0000 7f091e7fc640 1 -- 172.21.15.116:0/2592088110 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "389dcf96-e639-45ee-b53c-0efd01b1be3a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0920017070 con 0x7f092815b6d0 2024-01-20T08:30:36.822 INFO:teuthology.orchestra.run.smithi116.stdout:6 2024-01-20T08:30:36.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 msgr2=0x7f0928158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 0x7f0928158670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f092000ca80 tx=0x7f092000cf40 comp rx=0 tx=0).stop 2024-01-20T08:30:36.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 shutdown_connections 2024-01-20T08:30:36.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f092815b6d0 0x7f0928158670 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09280c58f0 0x7f092806a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 --2- 172.21.15.116:0/2592088110 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09280c5510 0x7f09281f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 >> 172.21.15.116:0/2592088110 conn(0x7f0928072810 msgr2=0x7f09280c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:36.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 shutdown_connections 2024-01-20T08:30:36.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.820+0000 7f092ff05640 1 -- 172.21.15.116:0/2592088110 wait complete. 2024-01-20T08:30:36.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 98176cde-6f9f-43be-96db-561e583ffacc 7 2024-01-20T08:30:36.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.929+0000 7f325b228640 1 Processor -- start 2024-01-20T08:30:36.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.929+0000 7f325b228640 1 -- start start 2024-01-20T08:30:36.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.930+0000 7f325b228640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32541571c0 0x7f32540c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.930+0000 7f325b228640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f325406e3f0 0x7f32540c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.930+0000 7f325b228640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3254156d30 con 0x7f3254150730 2024-01-20T08:30:36.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.930+0000 7f325b228640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3254156460 con 0x7f32541571c0 2024-01-20T08:30:36.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.930+0000 7f325b228640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325406ecf0 con 0x7f325406e3f0 2024-01-20T08:30:36.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.931+0000 7f32597c5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f325406e3f0 0x7f32540c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.931+0000 7f3258fc4640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3254150730 0x7f325406f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49996/0 (socket says 172.21.15.116:49996) 2024-01-20T08:30:36.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.931+0000 7f3258fc4640 1 -- 172.21.15.116:0/4224902832 learned_addr learned my addr 172.21.15.116:0/4224902832 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-01-20T08:30:36.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.931+0000 7f3253fff640 1 --2- 172.21.15.116:0/4224902832 >> v2:172.21.15.87:3300/0 conn(0x7f32541571c0 0x7f32540c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.933+0000 7f3253fff640 1 -- 172.21.15.116:0/4224902832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f325406e3f0 msgr2=0x7f32540c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:36.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.933+0000 7f3253fff640 1 --2- 172.21.15.116:0/4224902832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f325406e3f0 0x7f32540c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.933+0000 7f3253fff640 1 -- 172.21.15.116:0/4224902832 >> v1:172.21.15.116:6789/0 conn(0x7f3254150730 legacy=0x7f325406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.933+0000 7f3253fff640 1 -- 172.21.15.116:0/4224902832 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325406efd0 con 0x7f32541571c0 2024-01-20T08:30:36.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.935+0000 7f3253fff640 1 --2- 172.21.15.116:0/4224902832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f32540c6860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f32400097a0 tx=0x7f324002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf29ddcdaed7d0b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.939+0000 7f32537fe640 1 -- 172.21.15.116:0/4224902832 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240030610 con 0x7f32541571c0 2024-01-20T08:30:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.939+0000 7f32537fe640 1 -- 172.21.15.116:0/4224902832 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3240030770 con 0x7f32541571c0 2024-01-20T08:30:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.939+0000 7f32537fe640 1 -- 172.21.15.116:0/4224902832 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240030b00 con 0x7f32541571c0 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.939+0000 7f325b228640 1 -- 172.21.15.116:0/4224902832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 msgr2=0x7f32540c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.939+0000 7f325b228640 1 --2- 172.21.15.116:0/4224902832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f32540c6860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f32400097a0 tx=0x7f324002b960 comp rx=0 tx=0).stop 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.940+0000 7f325b228640 1 -- 172.21.15.116:0/4224902832 shutdown_connections 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.940+0000 7f325b228640 1 --2- 172.21.15.116:0/4224902832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f325406e3f0 0x7f32540c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.940+0000 7f325b228640 1 --2- 172.21.15.116:0/4224902832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f32540c6860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.940+0000 7f325b228640 1 -- 172.21.15.116:0/4224902832 >> 172.21.15.116:0/4224902832 conn(0x7f325406b6c0 msgr2=0x7f325406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.940+0000 7f325b228640 1 -- 172.21.15.116:0/4224902832 shutdown_connections 2024-01-20T08:30:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.940+0000 7f325b228640 1 -- 172.21.15.116:0/4224902832 wait complete. 2024-01-20T08:30:36.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.941+0000 7f325b228640 1 Processor -- start 2024-01-20T08:30:36.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.941+0000 7f325b228640 1 -- start start 2024-01-20T08:30:36.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f325b228640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f325406e3f0 0x7f32541ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f325b228640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3254150730 0x7f325414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f325b228640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f3254159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:36.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f325b228640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254146d30 con 0x7f32541571c0 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f325b228640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541efd10 con 0x7f325406e3f0 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f325b228640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3254146520 con 0x7f3254150730 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f3253fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3254150730 0x7f325414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f32597c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f3254159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.942+0000 7f3258fc4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f325406e3f0 0x7f32541ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.943+0000 7f32597c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f3254159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.116:54746/0 (socket says 172.21.15.116:54746) 2024-01-20T08:30:36.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.943+0000 7f32597c5640 1 -- 172.21.15.116:0/3422330987 learned_addr learned my addr 172.21.15.116:0/3422330987 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-01-20T08:30:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.945+0000 7f32597c5640 1 -- 172.21.15.116:0/3422330987 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3254150730 msgr2=0x7f325414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.945+0000 7f32597c5640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3254150730 0x7f325414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.945+0000 7f32597c5640 1 -- 172.21.15.116:0/3422330987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f325406e3f0 msgr2=0x7f32541ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.945+0000 7f32597c5640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f325406e3f0 0x7f32541ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.945+0000 7f32597c5640 1 -- 172.21.15.116:0/3422330987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240009400 con 0x7f32541571c0 2024-01-20T08:30:36.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.947+0000 7f32597c5640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f3254159460 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f324c00efc0 tx=0x7f324c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:36.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.951+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c014480 con 0x7f32541571c0 2024-01-20T08:30:36.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.951+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c00f040 con 0x7f32541571c0 2024-01-20T08:30:36.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.951+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c014820 con 0x7f32541571c0 2024-01-20T08:30:36.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.951+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325414b990 con 0x7f32541571c0 2024-01-20T08:30:36.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.951+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3254146770 con 0x7f32541571c0 2024-01-20T08:30:36.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.952+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32540c2ab0 con 0x7f32541571c0 2024-01-20T08:30:36.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.953+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f324c014980 con 0x7f32541571c0 2024-01-20T08:30:36.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:36.953+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f324c027dc0 con 0x7f32541571c0 2024-01-20T08:30:37.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.053+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f324c018020 con 0x7f32541571c0 2024-01-20T08:30:37.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.125+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "98176cde-6f9f-43be-96db-561e583ffacc", "id": 7} v 0) v1 -- 0x7f3254074290 con 0x7f32541571c0 2024-01-20T08:30:37.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.162+0000 7f32517fa640 1 -- 172.21.15.116:0/3422330987 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98176cde-6f9f-43be-96db-561e583ffacc", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f324c014d00 con 0x7f32541571c0 2024-01-20T08:30:37.165 INFO:teuthology.orchestra.run.smithi116.stdout:7 2024-01-20T08:30:37.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 msgr2=0x7f3254159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f3254159460 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f324c00efc0 tx=0x7f324c00c490 comp rx=0 tx=0).stop 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 shutdown_connections 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32541571c0 0x7f3254159460 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3254150730 0x7f325414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 --2- 172.21.15.116:0/3422330987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f325406e3f0 0x7f32541ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 >> 172.21.15.116:0/3422330987 conn(0x7f325406b6c0 msgr2=0x7f325406f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 shutdown_connections 2024-01-20T08:30:37.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-01-20T08:30:37.163+0000 7f325b228640 1 -- 172.21.15.116:0/3422330987 wait complete. 2024-01-20T08:30:37.178 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-20T08:30:37.178 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 0 2024-01-20T08:30:37.180 INFO:tasks.ceph.osd.0:Started 2024-01-20T08:30:37.180 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-20T08:30:37.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 1 2024-01-20T08:30:37.185 INFO:tasks.ceph.osd.1:Started 2024-01-20T08:30:37.185 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-20T08:30:37.185 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 2 2024-01-20T08:30:37.188 INFO:tasks.ceph.osd.2:Started 2024-01-20T08:30:37.188 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-20T08:30:37.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 3 2024-01-20T08:30:37.190 INFO:tasks.ceph.osd.3:Started 2024-01-20T08:30:37.190 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-20T08:30:37.190 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 4 2024-01-20T08:30:37.192 INFO:tasks.ceph.osd.4:Started 2024-01-20T08:30:37.192 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-20T08:30:37.192 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 5 2024-01-20T08:30:37.194 INFO:tasks.ceph.osd.5:Started 2024-01-20T08:30:37.194 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-20T08:30:37.195 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 6 2024-01-20T08:30:37.196 INFO:tasks.ceph.osd.6:Started 2024-01-20T08:30:37.197 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-20T08:30:37.197 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 7 2024-01-20T08:30:37.198 INFO:tasks.ceph.osd.7:Started 2024-01-20T08:30:37.199 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:30:37.317 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:37.314+0000 7f99e1217040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-20T08:30:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.320+0000 7f65e60b4640 1 Processor -- start 2024-01-20T08:30:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.320+0000 7f65e60b4640 1 -- start start 2024-01-20T08:30:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.321+0000 7f65e60b4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f65e0153840 0x7f65e00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.321+0000 7f65e60b4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f65e00cb110 0x7f65e00ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.321+0000 7f65e60b4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e0072a70 con 0x7f65e00c9820 2024-01-20T08:30:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.321+0000 7f65e60b4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e0153c20 con 0x7f65e00cb110 2024-01-20T08:30:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.321+0000 7f65e60b4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e015a430 con 0x7f65e0153840 2024-01-20T08:30:37.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.323+0000 7f65deffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f65e00cb110 0x7f65e00ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.324+0000 7f65deffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f65e00cb110 0x7f65e00ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47664/0 (socket says 172.21.15.87:47664) 2024-01-20T08:30:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.324+0000 7f65deffd640 1 -- 172.21.15.87:0/1476532790 learned_addr learned my addr 172.21.15.87:0/1476532790 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.325+0000 7f65df7fe640 1 --2- 172.21.15.87:0/1476532790 >> v2:172.21.15.116:3300/0 conn(0x7f65e0153840 0x7f65e00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:37.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.327+0000 7f65deffd640 1 -- 172.21.15.87:0/1476532790 >> v2:172.21.15.116:3300/0 conn(0x7f65e0153840 msgr2=0x7f65e00cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.327+0000 7f65deffd640 1 --2- 172.21.15.87:0/1476532790 >> v2:172.21.15.116:3300/0 conn(0x7f65e0153840 0x7f65e00cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.327+0000 7f65deffd640 1 -- 172.21.15.87:0/1476532790 >> v1:172.21.15.116:6789/0 conn(0x7f65e00c9820 legacy=0x7f65e015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.327+0000 7f65deffd640 1 -- 172.21.15.87:0/1476532790 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e00cb610 con 0x7f65e00cb110 2024-01-20T08:30:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.330+0000 7f65deffd640 1 --2- 172.21.15.87:0/1476532790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00cb110 0x7f65e00ca090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65e00cc560 tx=0x7f65c802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8796b021504ca0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.334+0000 7f65de7fc640 1 -- 172.21.15.87:0/1476532790 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c802f850 con 0x7f65e00cb110 2024-01-20T08:30:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.334+0000 7f65de7fc640 1 -- 172.21.15.87:0/1476532790 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c802f9b0 con 0x7f65e00cb110 2024-01-20T08:30:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.334+0000 7f65de7fc640 1 -- 172.21.15.87:0/1476532790 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c802fd40 con 0x7f65e00cb110 2024-01-20T08:30:37.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 -- 172.21.15.87:0/1476532790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00cb110 msgr2=0x7f65e00ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:37.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 --2- 172.21.15.87:0/1476532790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00cb110 0x7f65e00ca090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65e00cc560 tx=0x7f65c802ba30 comp rx=0 tx=0).stop 2024-01-20T08:30:37.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 -- 172.21.15.87:0/1476532790 shutdown_connections 2024-01-20T08:30:37.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 --2- 172.21.15.87:0/1476532790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00cb110 0x7f65e00ca090 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 --2- 172.21.15.87:0/1476532790 >> v2:172.21.15.116:3300/0 conn(0x7f65e0153840 0x7f65e00cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 -- 172.21.15.87:0/1476532790 >> 172.21.15.87:0/1476532790 conn(0x7f65e008b5c0 msgr2=0x7f65e0071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 -- 172.21.15.87:0/1476532790 shutdown_connections 2024-01-20T08:30:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.335+0000 7f65e60b4640 1 -- 172.21.15.87:0/1476532790 wait complete. 2024-01-20T08:30:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.336+0000 7f65e60b4640 1 Processor -- start 2024-01-20T08:30:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.336+0000 7f65e60b4640 1 -- start start 2024-01-20T08:30:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.337+0000 7f65e60b4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00c9820 0x7f65e01ef490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.337+0000 7f65e60b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 0x7f65e00c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.337+0000 7f65e60b4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e0153840 0x7f65e01f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.337+0000 7f65e60b4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00c84c0 con 0x7f65e00c9820 2024-01-20T08:30:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.337+0000 7f65e60b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00c75d0 con 0x7f65e00cb110 2024-01-20T08:30:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.337+0000 7f65e60b4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00cacd0 con 0x7f65e0153840 2024-01-20T08:30:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.338+0000 7f65deffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 0x7f65e00c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.338+0000 7f65dffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e0153840 0x7f65e01f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.338+0000 7f65deffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 0x7f65e00c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:44950/0 (socket says 172.21.15.87:44950) 2024-01-20T08:30:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.338+0000 7f65deffd640 1 -- 172.21.15.87:0/2467695425 learned_addr learned my addr 172.21.15.87:0/2467695425 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.339+0000 7f65df7fe640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00c9820 0x7f65e01ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.341+0000 7f65deffd640 1 -- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e0153840 msgr2=0x7f65e01f0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.341+0000 7f65deffd640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e0153840 0x7f65e01f0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.341+0000 7f65deffd640 1 -- 172.21.15.87:0/2467695425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00c9820 msgr2=0x7f65e01ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.341+0000 7f65deffd640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00c9820 0x7f65e01ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.341+0000 7f65deffd640 1 -- 172.21.15.87:0/2467695425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80094d0 con 0x7f65e00cb110 2024-01-20T08:30:37.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.344+0000 7f65deffd640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 0x7f65e00c6be0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000c00 tx=0x7f65c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.347+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c803b040 con 0x7f65e00cb110 2024-01-20T08:30:37.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.347+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8037c60 con 0x7f65e00cb110 2024-01-20T08:30:37.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.347+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8042480 con 0x7f65e00cb110 2024-01-20T08:30:37.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.347+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e00c88e0 con 0x7f65e00cb110 2024-01-20T08:30:37.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.348+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e00c7b20 con 0x7f65e00cb110 2024-01-20T08:30:37.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.348+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e00b25a0 con 0x7f65e00cb110 2024-01-20T08:30:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.351+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f65c8042960 con 0x7f65e00cb110 2024-01-20T08:30:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.351+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f65c804ccb0 con 0x7f65e00cb110 2024-01-20T08:30:37.376 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:37.373+0000 7efc5f64c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-20T08:30:37.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.446+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f65c8042c10 con 0x7f65e00cb110 2024-01-20T08:30:37.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.508+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65e0074290 con 0x7f65e00cb110 2024-01-20T08:30:37.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.554+0000 7f65d7fff640 1 -- 172.21.15.87:0/2467695425 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f65c8042e40 con 0x7f65e00cb110 2024-01-20T08:30:37.557 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:30:37.558 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":10,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:30:37.131717+0000","last_up_change":"0.000000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 msgr2=0x7f65e00c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 0x7f65e00c6be0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000c00 tx=0x7f65c80046c0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 shutdown_connections 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e0153840 0x7f65e01f0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e00cb110 0x7f65e00c6be0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 --2- 172.21.15.87:0/2467695425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00c9820 0x7f65e01ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 >> 172.21.15.87:0/2467695425 conn(0x7f65e008b5c0 msgr2=0x7f65e01459b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 shutdown_connections 2024-01-20T08:30:37.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:37.555+0000 7f65e60b4640 1 -- 172.21.15.87:0/2467695425 wait complete. 2024-01-20T08:30:37.569 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-20T08:30:37.569 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-20T08:30:37.608 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:37.606+0000 7f99e1217040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-20T08:30:37.654 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:37.652+0000 7efc5f64c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-20T08:30:37.659 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:37.657+0000 7f99e1217040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-20T08:30:37.704 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:37.701+0000 7efc5f64c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-20T08:30:37.757 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:37.755+0000 7f99e1217040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-20T08:30:37.792 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-01-20T08:30:37.789+0000 7f74c6224640 -1 Falling back to public interface 2024-01-20T08:30:37.803 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-01-20T08:30:37.800+0000 7efc5f64c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-20T08:30:37.826 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-01-20T08:30:37.823+0000 7fd28e79a640 -1 Falling back to public interface 2024-01-20T08:30:37.828 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-01-20T08:30:37.825+0000 7f8eef6a8640 -1 Falling back to public interface 2024-01-20T08:30:37.831 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-01-20T08:30:37.828+0000 7fdc709bd640 -1 Falling back to public interface 2024-01-20T08:30:37.831 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T08:30:37.828+0000 7f5e8ec89640 -1 Falling back to public interface 2024-01-20T08:30:37.832 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T08:30:37.829+0000 7f5e8dc32640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6820/0: (98) Address already in use 2024-01-20T08:30:37.841 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-01-20T08:30:37.838+0000 7f3b00424640 -1 Falling back to public interface 2024-01-20T08:30:37.849 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-01-20T08:30:37.846+0000 7f2715611640 -1 Falling back to public interface 2024-01-20T08:30:37.850 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-01-20T08:30:37.847+0000 7fc3e8c06640 -1 Falling back to public interface 2024-01-20T08:30:38.237 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:38.234+0000 7f99db613640 -1 mgr handle_mgr_map I was active but no longer am 2024-01-20T08:30:38.375 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-20T08:30:38.464 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.461+0000 7fdd56016640 1 Processor -- start 2024-01-20T08:30:38.465 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.462+0000 7fdd56016640 1 -- start start 2024-01-20T08:30:38.465 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.462+0000 7fdd56016640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdd5006a590 0x7fdd500c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:38.465 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.462+0000 7fdd56016640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd50159a00 con 0x7fdd50070320 2024-01-20T08:30:38.465 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.462+0000 7fdd56016640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd50159ce0 con 0x7fdd501539b0 2024-01-20T08:30:38.465 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.462+0000 7fdd56016640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5015a860 con 0x7fdd5006a590 2024-01-20T08:30:38.466 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.463+0000 7fdd4ffff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdd5006a590 0x7fdd500c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:38.466 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.463+0000 7fdd4f7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdd501539b0 0x7fdd5006fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:52364/0 (socket says 172.21.15.87:52364) 2024-01-20T08:30:38.466 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.463+0000 7fdd4f7fe640 1 -- 172.21.15.87:0/4142590342 learned_addr learned my addr 172.21.15.87:0/4142590342 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:30:38.468 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.465+0000 7fdd4e7fc640 1 -- 172.21.15.87:0/4142590342 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281976805 0 0) 0x7fdd50159ce0 con 0x7fdd501539b0 2024-01-20T08:30:38.468 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.465+0000 7fdd4e7fc640 1 -- 172.21.15.87:0/4142590342 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd38003520 con 0x7fdd501539b0 2024-01-20T08:30:38.472 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.469+0000 7fdd4ffff640 1 -- 172.21.15.87:0/4142590342 >> v1:172.21.15.116:6789/0 conn(0x7fdd501539b0 legacy=0x7fdd5006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:38.472 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.469+0000 7fdd4ffff640 1 -- 172.21.15.87:0/4142590342 >> v1:172.21.15.87:6789/0 conn(0x7fdd50070320 legacy=0x7fdd500c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:38.472 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.469+0000 7fdd4ffff640 1 -- 172.21.15.87:0/4142590342 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50159fc0 con 0x7fdd5006a590 2024-01-20T08:30:38.475 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.473+0000 7fdd4ffff640 1 --2- 172.21.15.87:0/4142590342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd500c2ac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009d90 tx=0x7fdd4802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1053fcfa32bb60aa server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:38.480 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.477+0000 7fdd4e7fc640 1 -- 172.21.15.87:0/4142590342 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4802cd00 con 0x7fdd5006a590 2024-01-20T08:30:38.480 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.477+0000 7fdd4e7fc640 1 -- 172.21.15.87:0/4142590342 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4802ce60 con 0x7fdd5006a590 2024-01-20T08:30:38.480 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.477+0000 7fdd4e7fc640 1 -- 172.21.15.87:0/4142590342 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48034660 con 0x7fdd5006a590 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 -- 172.21.15.87:0/4142590342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 msgr2=0x7fdd500c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 --2- 172.21.15.87:0/4142590342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd500c2ac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009d90 tx=0x7fdd4802bf50 comp rx=0 tx=0).stop 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 -- 172.21.15.87:0/4142590342 shutdown_connections 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 --2- 172.21.15.87:0/4142590342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd500c2ac0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 -- 172.21.15.87:0/4142590342 >> 172.21.15.87:0/4142590342 conn(0x7fdd500cbeb0 msgr2=0x7fdd500cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 -- 172.21.15.87:0/4142590342 shutdown_connections 2024-01-20T08:30:38.481 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.478+0000 7fdd56016640 1 -- 172.21.15.87:0/4142590342 wait complete. 2024-01-20T08:30:38.482 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.479+0000 7fdd56016640 1 Processor -- start 2024-01-20T08:30:38.482 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.479+0000 7fdd56016640 1 -- start start 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.480+0000 7fdd56016640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd5014b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.480+0000 7fdd56016640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 0x7fdd5014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.480+0000 7fdd56016640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 0x7fdd50145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.480+0000 7fdd56016640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015b210 con 0x7fdd5006a590 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.480+0000 7fdd56016640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5014a6d0 con 0x7fdd501539b0 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.480+0000 7fdd56016640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015af30 con 0x7fdd50070320 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd5014b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:38.484 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 0x7fdd50145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:38.485 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4effd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 0x7fdd5014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:38.485 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd5014b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47722/0 (socket says 172.21.15.87:47722) 2024-01-20T08:30:38.485 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4effd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 0x7fdd5014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49646/0 (socket says 172.21.15.87:49646) 2024-01-20T08:30:38.485 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 0x7fdd50145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:44990/0 (socket says 172.21.15.87:44990) 2024-01-20T08:30:38.485 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.481+0000 7fdd4f7fe640 1 -- 172.21.15.87:0/46555344 learned_addr learned my addr 172.21.15.87:0/46555344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:38.487 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4ffff640 1 -- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 msgr2=0x7fdd5014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:38.487 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4ffff640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 0x7fdd5014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:38.487 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4ffff640 1 -- 172.21.15.87:0/46555344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 msgr2=0x7fdd5014b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:38.487 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4ffff640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd5014b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:38.487 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4ffff640 1 -- 172.21.15.87:0/46555344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009a70 con 0x7fdd501539b0 2024-01-20T08:30:38.487 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4effd640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 0x7fdd5014ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:30:38.488 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.484+0000 7fdd4f7fe640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd5014b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:30:38.490 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.487+0000 7fdd4ffff640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 0x7fdd50145580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdd5006f5f0 tx=0x7fdd48004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:38.496 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.493+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4803d600 con 0x7fdd501539b0 2024-01-20T08:30:38.496 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.493+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4803d760 con 0x7fdd501539b0 2024-01-20T08:30:38.496 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.493+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd500cb4e0 con 0x7fdd501539b0 2024-01-20T08:30:38.496 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.493+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4803db10 con 0x7fdd501539b0 2024-01-20T08:30:38.496 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.493+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd500cbaa0 con 0x7fdd501539b0 2024-01-20T08:30:38.497 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.494+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5006c670 con 0x7fdd501539b0 2024-01-20T08:30:38.603 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.600+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 2) v1 ==== 44620+0+0 (secure 0 0 0) 0x7fdd4803b040 con 0x7fdd501539b0 2024-01-20T08:30:38.604 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.601+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fdd48032340 con 0x7fdd501539b0 2024-01-20T08:30:38.609 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.606+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fdd48009450 con 0x7fdd501539b0 2024-01-20T08:30:38.649 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:38.646+0000 7f1d6784b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-20T08:30:38.672 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.669+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5014a840 con 0x7fdd501539b0 2024-01-20T08:30:38.698 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:38.695+0000 7f1d6784b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-20T08:30:38.700 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.697+0000 7fdd37fff640 1 -- 172.21.15.87:0/46555344 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fdd4804d070 con 0x7fdd501539b0 2024-01-20T08:30:38.700 INFO:teuthology.misc.health.smithi087.stdout: 2024-01-20T08:30:38.700 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":10,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:30:37.131717+0000","last_up_change":"0.000000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.697+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 msgr2=0x7fdd50145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.697+0000 7fdd56016640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 0x7fdd50145580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdd5006f5f0 tx=0x7fdd48004a40 comp rx=0 tx=0).stop 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 shutdown_connections 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd501539b0 0x7fdd50145580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd50070320 0x7fdd5014ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 --2- 172.21.15.87:0/46555344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5006a590 0x7fdd5014b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 >> 172.21.15.87:0/46555344 conn(0x7fdd500cbeb0 msgr2=0x7fdd5014e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 shutdown_connections 2024-01-20T08:30:38.701 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:38.698+0000 7fdd56016640 1 -- 172.21.15.87:0/46555344 wait complete. 2024-01-20T08:30:38.711 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-20T08:30:39.092 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.089+0000 7f1d6784b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-20T08:30:39.136 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.134+0000 7f1d6784b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-20T08:30:39.181 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.178+0000 7f1d6784b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-20T08:30:39.235 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.232+0000 7f1d6784b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-20T08:30:39.276 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.273+0000 7f1d6784b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-20T08:30:39.290 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:30:39.293 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:30:39.378 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.375+0000 7f1d6784b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-20T08:30:39.431 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.429+0000 7f1d6784b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-20T08:30:39.475 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.472+0000 7f1d6784b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-20T08:30:39.524 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.521+0000 7f1d6784b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-20T08:30:39.566 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.563+0000 7f1d6784b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-20T08:30:39.891 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:39.888+0000 7f1d6784b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-20T08:30:40.023 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.020+0000 7f1d6784b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-20T08:30:40.115 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.112+0000 7f1d6784b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-20T08:30:40.157 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.154+0000 7f1d6784b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-20T08:30:40.252 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.249+0000 7f1d6784b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-20T08:30:40.354 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.351+0000 7f1d6784b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-20T08:30:40.490 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.487+0000 7f1d6784b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-20T08:30:40.534 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.531+0000 7f1d6784b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-20T08:30:40.719 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:40.716+0000 7f1d6784b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-20T08:30:41.285 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-01-20T08:30:41.282+0000 7f8eef6a8640 -1 osd.7 0 log_to_monitors true 2024-01-20T08:30:41.329 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-01-20T08:30:41.326+0000 7fd28e79a640 -1 osd.2 0 log_to_monitors true 2024-01-20T08:30:41.337 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-01-20T08:30:41.334+0000 7f74c6224640 -1 osd.0 0 log_to_monitors true 2024-01-20T08:30:41.347 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:41.345+0000 7f1d6784b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-20T08:30:41.353 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-01-20T08:30:41.350+0000 7f3b00424640 -1 osd.4 0 log_to_monitors true 2024-01-20T08:30:41.353 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-01-20T08:30:41.350+0000 7fdc709bd640 -1 osd.3 0 log_to_monitors true 2024-01-20T08:30:41.358 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T08:30:41.355+0000 7f5e8ec89640 -1 osd.1 0 log_to_monitors true 2024-01-20T08:30:41.366 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-01-20T08:30:41.363+0000 7f2715611640 -1 osd.5 0 log_to_monitors true 2024-01-20T08:30:41.366 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-01-20T08:30:41.364+0000 7fc3e8c06640 -1 osd.6 0 log_to_monitors true 2024-01-20T08:30:41.628 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:41.625+0000 7f1d6784b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-20T08:30:41.679 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:41.676+0000 7f1d6784b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-20T08:30:41.779 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:41.776+0000 7f1d6784b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-20T08:30:42.837 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-01-20T08:30:42.834+0000 7fdc6d161640 -1 osd.3 0 waiting for initial osdmap 2024-01-20T08:30:42.854 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-01-20T08:30:42.851+0000 7f8eeb638640 -1 osd.7 0 waiting for initial osdmap 2024-01-20T08:30:42.886 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-01-20T08:30:42.883+0000 7fdc64774640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:42.888 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-01-20T08:30:42.885+0000 7f8ee345f640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:43.676 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:43.673+0000 7f1d08bb2640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-20T08:30:43.753 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T08:30:43.750+0000 7f1d08bb2640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-20T08:30:44.805 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-01-20T08:30:44.802+0000 7f3afc3b4640 -1 osd.4 0 waiting for initial osdmap 2024-01-20T08:30:44.935 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-01-20T08:30:44.932+0000 7f3af41db640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:44.949 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-01-20T08:30:44.946+0000 7fd28a72a640 -1 osd.2 0 waiting for initial osdmap 2024-01-20T08:30:44.969 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T08:30:44.966+0000 7f5e8ac19640 -1 osd.1 0 waiting for initial osdmap 2024-01-20T08:30:44.996 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-01-20T08:30:44.993+0000 7f2711835640 -1 osd.5 0 waiting for initial osdmap 2024-01-20T08:30:45.035 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T08:30:45.032+0000 7f5e82a40640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:45.035 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-01-20T08:30:45.032+0000 7fd282551640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:45.082 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-01-20T08:30:45.079+0000 7fc3e4b96640 -1 osd.6 0 waiting for initial osdmap 2024-01-20T08:30:45.103 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-01-20T08:30:45.100+0000 7f2708e48640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:45.184 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-01-20T08:30:45.181+0000 7fc3dc9bd640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:45.517 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-20T08:30:45.614 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.610+0000 7f52df3b6640 1 Processor -- start 2024-01-20T08:30:45.614 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.610+0000 7f52df3b6640 1 -- start start 2024-01-20T08:30:45.615 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.611+0000 7f52df3b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d80c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:45.615 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.611+0000 7f52df3b6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f52d80c9d40 0x7f52d80ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:45.615 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.611+0000 7f52df3b6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d8153d40 con 0x7f52d815cb20 2024-01-20T08:30:45.615 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.611+0000 7f52df3b6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d80b27b0 con 0x7f52d80c9d40 2024-01-20T08:30:45.615 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.611+0000 7f52df3b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d815d6c0 con 0x7f52d8153960 2024-01-20T08:30:45.615 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.611+0000 7f52dc951640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f52d80c9d40 0x7f52d80ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:45.616 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.612+0000 7f52dc951640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f52d80c9d40 0x7f52d80ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56562/0 (socket says 172.21.15.87:56562) 2024-01-20T08:30:45.616 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.612+0000 7f52dc951640 1 -- 172.21.15.87:0/2093043066 learned_addr learned my addr 172.21.15.87:0/2093043066 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:45.618 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.614+0000 7f52dc951640 1 -- 172.21.15.87:0/2093043066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 msgr2=0x7f52d80c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:45.618 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.614+0000 7f52dc951640 1 --2- 172.21.15.87:0/2093043066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d80c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:45.618 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.614+0000 7f52dc951640 1 -- 172.21.15.87:0/2093043066 >> v1:172.21.15.116:6789/0 conn(0x7f52d815cb20 legacy=0x7f52d815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:45.618 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.614+0000 7f52dc951640 1 -- 172.21.15.87:0/2093043066 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80cb100 con 0x7f52d80c9d40 2024-01-20T08:30:45.619 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.616+0000 7f52dc951640 1 --2- 172.21.15.87:0/2093043066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 0x7f52d80ca160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52c80098a0 tx=0x7f52c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e79871b3f44e3d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:45.623 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.620+0000 7f52cffff640 1 -- 172.21.15.87:0/2093043066 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8031640 con 0x7f52d80c9d40 2024-01-20T08:30:45.623 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.620+0000 7f52cffff640 1 -- 172.21.15.87:0/2093043066 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c80317a0 con 0x7f52d80c9d40 2024-01-20T08:30:45.624 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.620+0000 7f52cffff640 1 -- 172.21.15.87:0/2093043066 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8031b30 con 0x7f52d80c9d40 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 -- 172.21.15.87:0/2093043066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 msgr2=0x7f52d80ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 --2- 172.21.15.87:0/2093043066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 0x7f52d80ca160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52c80098a0 tx=0x7f52c802b8f0 comp rx=0 tx=0).stop 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 -- 172.21.15.87:0/2093043066 shutdown_connections 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 --2- 172.21.15.87:0/2093043066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 0x7f52d80ca160 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 --2- 172.21.15.87:0/2093043066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d80c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 -- 172.21.15.87:0/2093043066 >> 172.21.15.87:0/2093043066 conn(0x7f52d808afe0 msgr2=0x7f52d808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 -- 172.21.15.87:0/2093043066 shutdown_connections 2024-01-20T08:30:45.625 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.621+0000 7f52df3b6640 1 -- 172.21.15.87:0/2093043066 wait complete. 2024-01-20T08:30:45.626 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.622+0000 7f52df3b6640 1 Processor -- start 2024-01-20T08:30:45.626 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.622+0000 7f52df3b6640 1 -- start start 2024-01-20T08:30:45.627 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.623+0000 7f52df3b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 0x7f52d8145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:45.627 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.623+0000 7f52df3b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d8146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:45.627 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.623+0000 7f52df3b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d815cb20 0x7f52d8146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:45.627 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.623+0000 7f52df3b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814f2c0 con 0x7f52d80c9d40 2024-01-20T08:30:45.627 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.623+0000 7f52df3b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d80cdd60 con 0x7f52d815cb20 2024-01-20T08:30:45.627 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.623+0000 7f52df3b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d80cd980 con 0x7f52d8153960 2024-01-20T08:30:45.634 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.631+0000 7f52dc951640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d8146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:45.635 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.632+0000 7f52dc951640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d8146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:35474/0 (socket says 172.21.15.87:35474) 2024-01-20T08:30:45.635 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.632+0000 7f52dc951640 1 -- 172.21.15.87:0/4120871578 learned_addr learned my addr 172.21.15.87:0/4120871578 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:45.640 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.636+0000 7f52dc951640 1 -- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d815cb20 msgr2=0x7f52d8146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:45.640 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.636+0000 7f52dc951640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d815cb20 0x7f52d8146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:45.640 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.636+0000 7f52dc951640 1 -- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 msgr2=0x7f52d8145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:45.640 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.636+0000 7f52dc951640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 0x7f52d8145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:45.640 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.636+0000 7f52dc951640 1 -- 172.21.15.87:0/4120871578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c80094d0 con 0x7f52d8153960 2024-01-20T08:30:45.643 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.639+0000 7f52dc951640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d8146270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f52c80098a0 tx=0x7f52c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:45.648 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.644+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c803ac60 con 0x7f52d8153960 2024-01-20T08:30:45.648 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.644+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c803adc0 con 0x7f52d8153960 2024-01-20T08:30:45.648 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.644+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d814e6b0 con 0x7f52d8153960 2024-01-20T08:30:45.648 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.644+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8044630 con 0x7f52d8153960 2024-01-20T08:30:45.648 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.644+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d814ec70 con 0x7f52d8153960 2024-01-20T08:30:45.649 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.646+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d80b4b70 con 0x7f52d8153960 2024-01-20T08:30:45.667 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.664+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 6) v1 ==== 89142+0+0 (secure 0 0 0) 0x7f52c802f050 con 0x7f52d8153960 2024-01-20T08:30:45.668 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.664+0000 7f52cdffb640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f52bc06ae70 0x7f52bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:45.668 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.664+0000 7f52dd152640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f52bc06ae70 0x7f52bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:45.668 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.665+0000 7f52dd152640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f52bc06ae70 0x7f52bc06d330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009710 tx=0x7f52d4006c90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:45.669 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.666+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2791+0+0 (secure 0 0 0) 0x7f52c807ab30 con 0x7f52d8153960 2024-01-20T08:30:45.735 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-01-20T08:30:45.732+0000 7f74c29c8640 -1 osd.0 0 waiting for initial osdmap 2024-01-20T08:30:45.920 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:45.916+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f52c8076770 con 0x7f52d8153960 2024-01-20T08:30:45.966 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-01-20T08:30:45.962+0000 7f74b9fdb640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-20T08:30:46.097 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.093+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d8153d40 con 0x7f52d8153960 2024-01-20T08:30:46.422 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.417+0000 7f52cdffb640 1 -- 172.21.15.87:0/4120871578 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+11757 (secure 0 0 0) 0x7f52d8153d40 con 0x7f52d8153960 2024-01-20T08:30:46.422 INFO:teuthology.misc.health.smithi087.stdout: 2024-01-20T08:30:46.422 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":14,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:30:45.183081+0000","last_up_change":"2024-01-20T08:30:45.183081+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f52bc06ae70 msgr2=0x7f52bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f52bc06ae70 0x7f52bc06d330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009710 tx=0x7f52d4006c90 comp rx=0 tx=0).stop 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 msgr2=0x7f52d8146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d8146270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f52c80098a0 tx=0x7f52c80046c0 comp rx=0 tx=0).stop 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 shutdown_connections 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f52bc06ae70 0x7f52bc06d330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d815cb20 0x7f52d8146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:46.424 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52d8153960 0x7f52d8146270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:46.425 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 --2- 172.21.15.87:0/4120871578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d80c9d40 0x7f52d8145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:46.425 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 >> 172.21.15.87:0/4120871578 conn(0x7f52d808afe0 msgr2=0x7f52d815c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:46.425 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 shutdown_connections 2024-01-20T08:30:46.425 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:46.420+0000 7f52df3b6640 1 -- 172.21.15.87:0/4120871578 wait complete. 2024-01-20T08:30:46.433 DEBUG:teuthology.misc:6 of 8 OSDs are up 2024-01-20T08:30:53.240 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-20T08:30:53.339 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.335+0000 7f24533e3640 1 Processor -- start 2024-01-20T08:30:53.339 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.335+0000 7f24533e3640 1 -- start start 2024-01-20T08:30:53.340 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.336+0000 7f24533e3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f244c153990 0x7f244c0c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:53.340 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.336+0000 7f24533e3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f244c0c74b0 0x7f244c15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:53.340 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.336+0000 7f24533e3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c15cc40 con 0x7f244c15d610 2024-01-20T08:30:53.340 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.336+0000 7f24533e3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c159650 con 0x7f244c0c74b0 2024-01-20T08:30:53.340 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.336+0000 7f24533e3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c153d70 con 0x7f244c153990 2024-01-20T08:30:53.341 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.337+0000 7f245097e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f244c0c74b0 0x7f244c15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:53.341 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.337+0000 7f245117f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f244c153990 0x7f244c0c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:53.341 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.337+0000 7f245117f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f244c153990 0x7f244c0c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41858/0 (socket says 172.21.15.87:41858) 2024-01-20T08:30:53.341 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.337+0000 7f245117f640 1 -- 172.21.15.87:0/1362286242 learned_addr learned my addr 172.21.15.87:0/1362286242 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:53.341 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.337+0000 7f245097e640 1 --2- 172.21.15.87:0/1362286242 >> v2:172.21.15.87:3300/0 conn(0x7f244c0c74b0 0x7f244c15d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56606/0 (socket says 172.21.15.87:56606) 2024-01-20T08:30:53.344 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.341+0000 7f245117f640 1 -- 172.21.15.87:0/1362286242 >> v2:172.21.15.87:3300/0 conn(0x7f244c0c74b0 msgr2=0x7f244c15d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:53.344 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.341+0000 7f245117f640 1 --2- 172.21.15.87:0/1362286242 >> v2:172.21.15.87:3300/0 conn(0x7f244c0c74b0 0x7f244c15d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.344 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.341+0000 7f245117f640 1 -- 172.21.15.87:0/1362286242 >> v1:172.21.15.87:6789/0 conn(0x7f244c15d610 legacy=0x7f244c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:53.344 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.341+0000 7f245117f640 1 -- 172.21.15.87:0/1362286242 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c15da40 con 0x7f244c153990 2024-01-20T08:30:53.346 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.342+0000 7f245117f640 1 --2- 172.21.15.87:0/1362286242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c153990 0x7f244c0c6f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f244c072dc0 tx=0x7f244802c010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d363315edcf34bc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:53.350 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.346+0000 7f2443fff640 1 -- 172.21.15.87:0/1362286242 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244802cbb0 con 0x7f244c153990 2024-01-20T08:30:53.351 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f2443fff640 1 -- 172.21.15.87:0/1362286242 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244802cd10 con 0x7f244c153990 2024-01-20T08:30:53.351 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f2443fff640 1 -- 172.21.15.87:0/1362286242 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448034610 con 0x7f244c153990 2024-01-20T08:30:53.351 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 -- 172.21.15.87:0/1362286242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c153990 msgr2=0x7f244c0c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:53.351 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 --2- 172.21.15.87:0/1362286242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c153990 0x7f244c0c6f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f244c072dc0 tx=0x7f244802c010 comp rx=0 tx=0).stop 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 -- 172.21.15.87:0/1362286242 shutdown_connections 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 --2- 172.21.15.87:0/1362286242 >> v2:172.21.15.87:3300/0 conn(0x7f244c0c74b0 0x7f244c15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 --2- 172.21.15.87:0/1362286242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c153990 0x7f244c0c6f70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 -- 172.21.15.87:0/1362286242 >> 172.21.15.87:0/1362286242 conn(0x7f244c06e800 msgr2=0x7f244c071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 -- 172.21.15.87:0/1362286242 shutdown_connections 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.348+0000 7f24533e3640 1 -- 172.21.15.87:0/1362286242 wait complete. 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 Processor -- start 2024-01-20T08:30:53.352 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 -- start start 2024-01-20T08:30:53.353 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 0x7f244c14fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:53.353 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c15d610 0x7f244c14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:53.353 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f244c14c820 0x7f244c15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:53.353 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c14ced0 con 0x7f244c153990 2024-01-20T08:30:53.354 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c14d7d0 con 0x7f244c15d610 2024-01-20T08:30:53.354 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.349+0000 7f24533e3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c14ec30 con 0x7f244c14c820 2024-01-20T08:30:53.354 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.350+0000 7f245117f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 0x7f244c14fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:53.354 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.350+0000 7f245097e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c15d610 0x7f244c14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:53.354 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.350+0000 7f245117f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 0x7f244c14fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56616/0 (socket says 172.21.15.87:56616) 2024-01-20T08:30:53.355 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.350+0000 7f245117f640 1 -- 172.21.15.87:0/1114616506 learned_addr learned my addr 172.21.15.87:0/1114616506 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:53.355 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.350+0000 7f2451980640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f244c14c820 0x7f244c15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:53.357 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.354+0000 7f245117f640 1 -- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f244c14c820 msgr2=0x7f244c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:53.357 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.354+0000 7f245117f640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f244c14c820 0x7f244c15c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.357 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.354+0000 7f245117f640 1 -- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c15d610 msgr2=0x7f244c14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:53.358 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.354+0000 7f245117f640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c15d610 0x7f244c14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.358 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.354+0000 7f245117f640 1 -- 172.21.15.87:0/1114616506 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448009b30 con 0x7f244c153990 2024-01-20T08:30:53.360 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.356+0000 7f245117f640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 0x7f244c14fa70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2448000f80 tx=0x7f2448004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:53.364 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.361+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244803e450 con 0x7f244c153990 2024-01-20T08:30:53.364 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.361+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244803e5d0 con 0x7f244c153990 2024-01-20T08:30:53.364 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.361+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c14ef10 con 0x7f244c153990 2024-01-20T08:30:53.364 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.361+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244803e980 con 0x7f244c153990 2024-01-20T08:30:53.364 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.361+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c14f4d0 con 0x7f244c153990 2024-01-20T08:30:53.364 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.361+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c0671d0 con 0x7f244c153990 2024-01-20T08:30:53.608 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.604+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f244803eae0 con 0x7f244c153990 2024-01-20T08:30:53.608 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.604+0000 7f2441ffb640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f243006af20 0x7f243006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:53.608 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.604+0000 7f245097e640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f243006af20 0x7f243006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:53.608 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.606+0000 7f245097e640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f243006af20 0x7f243006d3e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f244c0b1f30 tx=0x7f243c009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:53.621 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.618+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f244807b510 con 0x7f244c153990 2024-01-20T08:30:53.621 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.618+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2448077850 con 0x7f244c153990 2024-01-20T08:30:53.793 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.790+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f244c0ca6d0 con 0x7f244c153990 2024-01-20T08:30:53.901 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.898+0000 7f2441ffb640 1 -- 172.21.15.87:0/1114616506 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12851 (secure 0 0 0) 0x7f244807ae70 con 0x7f244c153990 2024-01-20T08:30:53.902 INFO:teuthology.misc.health.smithi087.stdout: 2024-01-20T08:30:53.902 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":16,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:30:47.867029+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:30:53.904 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f243006af20 msgr2=0x7f243006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:53.904 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f243006af20 0x7f243006d3e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f244c0b1f30 tx=0x7f243c009190 comp rx=0 tx=0).stop 2024-01-20T08:30:53.904 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 msgr2=0x7f244c14fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:53.904 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 0x7f244c14fa70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2448000f80 tx=0x7f2448004a40 comp rx=0 tx=0).stop 2024-01-20T08:30:53.904 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 shutdown_connections 2024-01-20T08:30:53.904 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f243006af20 0x7f243006d3e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.905 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f244c14c820 0x7f244c15c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.905 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244c15d610 0x7f244c14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.905 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 --2- 172.21.15.87:0/1114616506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f244c153990 0x7f244c14fa70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:53.905 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 >> 172.21.15.87:0/1114616506 conn(0x7f244c0c74b0 msgr2=0x7f244c157450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:53.905 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 shutdown_connections 2024-01-20T08:30:53.905 INFO:teuthology.misc.health.smithi087.stderr:2024-01-20T08:30:53.901+0000 7f24533e3640 1 -- 172.21.15.87:0/1114616506 wait complete. 2024-01-20T08:30:53.913 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-20T08:30:53.913 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-20T08:30:53.913 DEBUG:tasks.ceph_manager:running mds.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-20T08:30:53.914 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-20T08:30:53.914 DEBUG:teuthology.orchestra.run.smithi087:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i a 2024-01-20T08:30:53.916 INFO:tasks.ceph.mds.a:Started 2024-01-20T08:30:53.916 DEBUG:tasks.ceph_manager:running mds.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-20T08:30:53.916 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-20T08:30:53.916 DEBUG:teuthology.orchestra.run.smithi087:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i c 2024-01-20T08:30:53.918 INFO:tasks.ceph.mds.c:Started 2024-01-20T08:30:53.918 DEBUG:tasks.ceph_manager:running mds.e under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.e.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-20T08:30:53.918 INFO:tasks.ceph.mds.e:Restarting daemon 2024-01-20T08:30:53.918 DEBUG:teuthology.orchestra.run.smithi087:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i e 2024-01-20T08:30:53.920 INFO:tasks.ceph.mds.e:Started 2024-01-20T08:30:53.920 DEBUG:tasks.ceph_manager:running mds.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-20T08:30:53.920 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-20T08:30:53.921 DEBUG:teuthology.orchestra.run.smithi116:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i b 2024-01-20T08:30:53.923 INFO:tasks.ceph.mds.b:Started 2024-01-20T08:30:53.923 DEBUG:tasks.ceph_manager:running mds.d under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.d.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-20T08:30:53.923 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-20T08:30:53.924 DEBUG:teuthology.orchestra.run.smithi116:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i d 2024-01-20T08:30:53.926 INFO:tasks.ceph.mds.d:Started 2024-01-20T08:30:53.926 DEBUG:tasks.ceph_manager:running mds.f under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.f.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-20T08:30:53.926 INFO:tasks.ceph.mds.f:Restarting daemon 2024-01-20T08:30:53.926 DEBUG:teuthology.orchestra.run.smithi116:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i f 2024-01-20T08:30:53.928 INFO:tasks.ceph.mds.f:Started 2024-01-20T08:30:53.928 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-20T08:30:53.929 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:30:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.033+0000 7fecf7ca2640 1 Processor -- start 2024-01-20T08:30:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.033+0000 7fecf7ca2640 1 -- start start 2024-01-20T08:30:54.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.034+0000 7fecf7ca2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fecf01596e0 0x7fecf00c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.034+0000 7fecf7ca2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf015cc70 con 0x7fecf0153aa0 2024-01-20T08:30:54.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.034+0000 7fecf7ca2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf0153e80 con 0x7fecf00ca2d0 2024-01-20T08:30:54.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.034+0000 7fecf7ca2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf006b880 con 0x7fecf01596e0 2024-01-20T08:30:54.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.035+0000 7fecf623f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fecf01596e0 0x7fecf00c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.035+0000 7fecf623f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fecf01596e0 0x7fecf00c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41886/0 (socket says 172.21.15.87:41886) 2024-01-20T08:30:54.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.035+0000 7fecf623f640 1 -- 172.21.15.87:0/2321553595 learned_addr learned my addr 172.21.15.87:0/2321553595 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:54.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.039+0000 7fecf623f640 1 -- 172.21.15.87:0/2321553595 >> v1:172.21.15.116:6789/0 conn(0x7fecf00ca2d0 legacy=0x7fecf015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.039+0000 7fecf623f640 1 -- 172.21.15.87:0/2321553595 >> v1:172.21.15.87:6789/0 conn(0x7fecf0153aa0 legacy=0x7fecf00c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:54.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.039+0000 7fecf623f640 1 -- 172.21.15.87:0/2321553595 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf0159c40 con 0x7fecf01596e0 2024-01-20T08:30:54.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.043+0000 7fecf623f640 1 --2- 172.21.15.87:0/2321553595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf01596e0 0x7fecf00c88e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fece8009e50 tx=0x7fece802c010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b14bd7bb0f5f9bf server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:54.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.045+0000 7fecf4a3c640 1 -- 172.21.15.87:0/2321553595 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece802cbb0 con 0x7fecf01596e0 2024-01-20T08:30:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.049+0000 7fecf4a3c640 1 -- 172.21.15.87:0/2321553595 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece802cd10 con 0x7fecf01596e0 2024-01-20T08:30:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.049+0000 7fecf4a3c640 1 -- 172.21.15.87:0/2321553595 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8034610 con 0x7fecf01596e0 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 -- 172.21.15.87:0/2321553595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf01596e0 msgr2=0x7fecf00c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/2321553595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf01596e0 0x7fecf00c88e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fece8009e50 tx=0x7fece802c010 comp rx=0 tx=0).stop 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 -- 172.21.15.87:0/2321553595 shutdown_connections 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/2321553595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf01596e0 0x7fecf00c88e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 -- 172.21.15.87:0/2321553595 >> 172.21.15.87:0/2321553595 conn(0x7fecf0066ac0 msgr2=0x7fecf006ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 -- 172.21.15.87:0/2321553595 shutdown_connections 2024-01-20T08:30:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.050+0000 7fecf7ca2640 1 -- 172.21.15.87:0/2321553595 wait complete. 2024-01-20T08:30:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.051+0000 7fecf7ca2640 1 Processor -- start 2024-01-20T08:30:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.051+0000 7fecf7ca2640 1 -- start start 2024-01-20T08:30:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.052+0000 7fecf7ca2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf0153aa0 0x7fecf014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.052+0000 7fecf7ca2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 0x7fecf01f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.052+0000 7fecf7ca2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecf00c44c0 0x7fecf014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.052+0000 7fecf7ca2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf00cbff0 con 0x7fecf00c44c0 2024-01-20T08:30:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.052+0000 7fecf7ca2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf01f2d50 con 0x7fecf0153aa0 2024-01-20T08:30:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.052+0000 7fecf7ca2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf00c6760 con 0x7fecf01596e0 2024-01-20T08:30:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.053+0000 7fecf523d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 0x7fecf01f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.053+0000 7fecf5a3e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf0153aa0 0x7fecf014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.053+0000 7fecf523d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 0x7fecf01f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:35510/0 (socket says 172.21.15.87:35510) 2024-01-20T08:30:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.053+0000 7fecf523d640 1 -- 172.21.15.87:0/1212336495 learned_addr learned my addr 172.21.15.87:0/1212336495 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:54.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.055+0000 7fecf623f640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecf00c44c0 0x7fecf014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.059+0000 7fecf523d640 1 -- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf0153aa0 msgr2=0x7fecf014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.059+0000 7fecf523d640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf0153aa0 0x7fecf014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.059+0000 7fecf523d640 1 -- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecf00c44c0 msgr2=0x7fecf014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.059+0000 7fecf523d640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecf00c44c0 0x7fecf014b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.059+0000 7fecf523d640 1 -- 172.21.15.87:0/1212336495 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece8009b30 con 0x7fecf01596e0 2024-01-20T08:30:54.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.062+0000 7fecf523d640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 0x7fecf01f3170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fece000c400 tx=0x7fece000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.066+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00052d0 con 0x7fecf01596e0 2024-01-20T08:30:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.066+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece0005430 con 0x7fecf01596e0 2024-01-20T08:30:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.066+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0003830 con 0x7fecf01596e0 2024-01-20T08:30:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.066+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf01f2a00 con 0x7fecf01596e0 2024-01-20T08:30:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.066+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf00c6be0 con 0x7fecf01596e0 2024-01-20T08:30:54.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.067+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf01f2e90 con 0x7fecf01596e0 2024-01-20T08:30:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.086+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fece0003990 con 0x7fecf01596e0 2024-01-20T08:30:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.087+0000 7fece67fc640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fecd406aed0 0x7fecd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.087+0000 7fecf5a3e640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fecd406aed0 0x7fecd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.087+0000 7fecf5a3e640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fecd406aed0 0x7fecd406d390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecec006fd0 tx=0x7fecec009060 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.089+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fece0057850 con 0x7fecf01596e0 2024-01-20T08:30:54.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.344+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fece0053b90 con 0x7fecf01596e0 2024-01-20T08:30:54.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.529+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecf01f2e90 con 0x7fecf01596e0 2024-01-20T08:30:54.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.559+0000 7fece67fc640 1 -- 172.21.15.87:0/1212336495 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12851 (secure 0 0 0) 0x7fecf01f2e90 con 0x7fecf01596e0 2024-01-20T08:30:54.562 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:30:54.562 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:30:47.867029+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:30:54.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fecd406aed0 msgr2=0x7fecd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fecd406aed0 0x7fecd406d390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecec006fd0 tx=0x7fecec009060 comp rx=0 tx=0).stop 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 msgr2=0x7fecf01f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 0x7fecf01f3170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fece000c400 tx=0x7fece000c8c0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 shutdown_connections 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fecd406aed0 0x7fecd406d390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecf00c44c0 0x7fecf014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fecf01596e0 0x7fecf01f3170 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 --2- 172.21.15.87:0/1212336495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fecf0153aa0 0x7fecf014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 >> 172.21.15.87:0/1212336495 conn(0x7fecf00ca2d0 msgr2=0x7fecf01516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 shutdown_connections 2024-01-20T08:30:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.561+0000 7fecf7ca2640 1 -- 172.21.15.87:0/1212336495 wait complete. 2024-01-20T08:30:54.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-20T08:30:54.581 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:30:54.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.688+0000 7fb554896640 1 Processor -- start 2024-01-20T08:30:54.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.689+0000 7fb554896640 1 -- start start 2024-01-20T08:30:54.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.690+0000 7fb554896640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5501517f0 0x7fb5500b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.690+0000 7fb554896640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5500c9550 0x7fb5500ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.690+0000 7fb554896640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5500ca070 con 0x7fb5500c90e0 2024-01-20T08:30:54.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.690+0000 7fb554896640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5500b0520 con 0x7fb5500c9550 2024-01-20T08:30:54.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.690+0000 7fb554896640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb550151bd0 con 0x7fb5501517f0 2024-01-20T08:30:54.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.691+0000 7fb54e59c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5501517f0 0x7fb5500b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.691+0000 7fb54ed9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5500c9550 0x7fb5500ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.691+0000 7fb54ed9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5500c9550 0x7fb5500ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56646/0 (socket says 172.21.15.87:56646) 2024-01-20T08:30:54.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.691+0000 7fb54ed9d640 1 -- 172.21.15.87:0/3945734635 learned_addr learned my addr 172.21.15.87:0/3945734635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:54.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.694+0000 7fb54ed9d640 1 -- 172.21.15.87:0/3945734635 >> v2:172.21.15.116:3300/0 conn(0x7fb5501517f0 msgr2=0x7fb5500b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:54.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.694+0000 7fb54ed9d640 1 --2- 172.21.15.87:0/3945734635 >> v2:172.21.15.116:3300/0 conn(0x7fb5501517f0 0x7fb5500b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.694+0000 7fb54ed9d640 1 -- 172.21.15.87:0/3945734635 >> v1:172.21.15.87:6789/0 conn(0x7fb5500c90e0 legacy=0x7fb550157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:54.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.694+0000 7fb54ed9d640 1 -- 172.21.15.87:0/3945734635 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550157ef0 con 0x7fb5500c9550 2024-01-20T08:30:54.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.697+0000 7fb54ed9d640 1 --2- 172.21.15.87:0/3945734635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb5500ccd00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb55006b250 tx=0x7fb54802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e49926827308ee6 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:54.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.701+0000 7fb54d59a640 1 -- 172.21.15.87:0/3945734635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54802ca80 con 0x7fb5500c9550 2024-01-20T08:30:54.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.702+0000 7fb54d59a640 1 -- 172.21.15.87:0/3945734635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54802cbe0 con 0x7fb5500c9550 2024-01-20T08:30:54.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.703+0000 7fb554896640 1 -- 172.21.15.87:0/3945734635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 msgr2=0x7fb5500ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.703+0000 7fb554896640 1 --2- 172.21.15.87:0/3945734635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb5500ccd00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb55006b250 tx=0x7fb54802bee0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.704+0000 7fb554896640 1 -- 172.21.15.87:0/3945734635 shutdown_connections 2024-01-20T08:30:54.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.704+0000 7fb554896640 1 --2- 172.21.15.87:0/3945734635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb5500ccd00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.704+0000 7fb554896640 1 --2- 172.21.15.87:0/3945734635 >> v2:172.21.15.116:3300/0 conn(0x7fb5501517f0 0x7fb5500b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.704+0000 7fb554896640 1 -- 172.21.15.87:0/3945734635 >> 172.21.15.87:0/3945734635 conn(0x7fb55006a5b0 msgr2=0x7fb55006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:54.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.704+0000 7fb554896640 1 -- 172.21.15.87:0/3945734635 shutdown_connections 2024-01-20T08:30:54.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.704+0000 7fb554896640 1 -- 172.21.15.87:0/3945734635 wait complete. 2024-01-20T08:30:54.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.706+0000 7fb554896640 1 Processor -- start 2024-01-20T08:30:54.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.706+0000 7fb554896640 1 -- start start 2024-01-20T08:30:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb554896640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5500c90e0 0x7fb55006d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb554896640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb55014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb554896640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 0x7fb5500cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb554896640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500cb8e0 con 0x7fb5500c9550 2024-01-20T08:30:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb554896640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb550154dd0 con 0x7fb5501517f0 2024-01-20T08:30:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb554896640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb550154270 con 0x7fb5500c90e0 2024-01-20T08:30:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.707+0000 7fb54ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 0x7fb5500cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.708+0000 7fb54e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5500c90e0 0x7fb55006d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.708+0000 7fb54dd9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb55014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.708+0000 7fb54ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 0x7fb5500cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41908/0 (socket says 172.21.15.87:41908) 2024-01-20T08:30:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.708+0000 7fb54ed9d640 1 -- 172.21.15.87:0/1902755499 learned_addr learned my addr 172.21.15.87:0/1902755499 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:54.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.711+0000 7fb54ed9d640 1 -- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5500c90e0 msgr2=0x7fb55006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.711+0000 7fb54ed9d640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5500c90e0 0x7fb55006d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.711+0000 7fb54ed9d640 1 -- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 msgr2=0x7fb55014f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.711+0000 7fb54ed9d640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb55014f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.711+0000 7fb54ed9d640 1 -- 172.21.15.87:0/1902755499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548009b30 con 0x7fb5501517f0 2024-01-20T08:30:54.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.715+0000 7fb54ed9d640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 0x7fb5500cbd70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb548036040 tx=0x7fb548004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.719+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548009510 con 0x7fb5501517f0 2024-01-20T08:30:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.719+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548004250 con 0x7fb5501517f0 2024-01-20T08:30:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.719+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb550154530 con 0x7fb5501517f0 2024-01-20T08:30:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.720+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548004600 con 0x7fb5501517f0 2024-01-20T08:30:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.720+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb550154af0 con 0x7fb5501517f0 2024-01-20T08:30:54.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:54.721+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5500cda10 con 0x7fb5501517f0 2024-01-20T08:30:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.069+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb54803e450 con 0x7fb5501517f0 2024-01-20T08:30:55.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.070+0000 7fb53effd640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb53006af20 0x7fb53006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.070+0000 7fb54e59c640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb53006af20 0x7fb53006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:55.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.070+0000 7fb54e59c640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb53006af20 0x7fb53006d3e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb540009b40 tx=0x7fb540009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.074+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb54807b030 con 0x7fb5501517f0 2024-01-20T08:30:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.084+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb548077370 con 0x7fb5501517f0 2024-01-20T08:30:55.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.260+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5500ca500 con 0x7fb5501517f0 2024-01-20T08:30:55.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.282+0000 7fb53effd640 1 -- 172.21.15.87:0/1902755499 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb548043020 con 0x7fb5501517f0 2024-01-20T08:30:55.285 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:30:55.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb53006af20 msgr2=0x7fb53006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb53006af20 0x7fb53006d3e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb540009b40 tx=0x7fb540009190 comp rx=0 tx=0).stop 2024-01-20T08:30:55.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 msgr2=0x7fb5500cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 0x7fb5500cbd70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb548036040 tx=0x7fb548004a40 comp rx=0 tx=0).stop 2024-01-20T08:30:55.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 shutdown_connections 2024-01-20T08:30:55.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb53006af20 0x7fb53006d3e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5501517f0 0x7fb5500cbd70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5500c9550 0x7fb55014f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 --2- 172.21.15.87:0/1902755499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5500c90e0 0x7fb55006d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 >> 172.21.15.87:0/1902755499 conn(0x7fb55006a5b0 msgr2=0x7fb550155280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:55.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 shutdown_connections 2024-01-20T08:30:55.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.284+0000 7fb554896640 1 -- 172.21.15.87:0/1902755499 wait complete. 2024-01-20T08:30:55.300 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:30:55.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.421+0000 7f04675a9640 1 Processor -- start 2024-01-20T08:30:55.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.422+0000 7f04675a9640 1 -- start start 2024-01-20T08:30:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.423+0000 7f04675a9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f046006f580 0x7f046006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.423+0000 7f04675a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0460159710 0x7f0460159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.423+0000 7f04675a9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046015a0a0 con 0x7f04601539a0 2024-01-20T08:30:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.423+0000 7f04675a9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046015dce0 con 0x7f046006f580 2024-01-20T08:30:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.423+0000 7f04675a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046015d7b0 con 0x7f0460159710 2024-01-20T08:30:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.439+0000 7f0465345640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f04601539a0 0x7f04600c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42358/0 (socket says 172.21.15.87:42358) 2024-01-20T08:30:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.439+0000 7f0465345640 1 -- 172.21.15.87:0/4009536959 learned_addr learned my addr 172.21.15.87:0/4009536959 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:30:55.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.445+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151154527 0 0) 0x7f045c001110 con 0x7f04601539a0 2024-01-20T08:30:55.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.445+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f044c003520 con 0x7f04601539a0 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.450+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977918414 0 0) 0x7f044c003520 con 0x7f04601539a0 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.450+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0460159710 msgr2=0x7f0460159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.450+0000 7f0457fff640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0460159710 0x7f0460159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.450+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 >> v2:172.21.15.116:3300/0 conn(0x7f046006f580 msgr2=0x7f046006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.450+0000 7f0457fff640 1 --2- 172.21.15.87:0/4009536959 >> v2:172.21.15.116:3300/0 conn(0x7f046006f580 0x7f046006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600cc440 con 0x7f04601539a0 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4017552009 0 0) 0x7f045c001800 con 0x7f04601539a0 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 >> v1:172.21.15.87:6789/0 conn(0x7f04601539a0 legacy=0x7f04600c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f044c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0460159710 0x7f044c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c004e20 0x7f044c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046015dce0 con 0x7f044c003970 2024-01-20T08:30:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046015d7b0 con 0x7f044c004e20 2024-01-20T08:30:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.451+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0055e0 con 0x7f0460159710 2024-01-20T08:30:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.455+0000 7f0465345640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f044c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:55.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.458+0000 7f0465b46640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c004e20 0x7f044c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:55.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.459+0000 7f0465345640 1 -- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0460159710 msgr2=0x7f044c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:55.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.459+0000 7f0465345640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0460159710 0x7f044c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.459+0000 7f0465345640 1 -- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c004e20 msgr2=0x7f044c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:55.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.459+0000 7f0465345640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c004e20 0x7f044c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.459+0000 7f0465345640 1 -- 172.21.15.87:0/4009536959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c003520 con 0x7f044c003970 2024-01-20T08:30:55.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.462+0000 7f0465345640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f044c003d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c110 tx=0x7f044c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eeea1ad0dc8563 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:55.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.465+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c005ad0 con 0x7f044c003970 2024-01-20T08:30:55.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.469+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c005c30 con 0x7f044c003970 2024-01-20T08:30:55.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.469+0000 7f0457fff640 1 -- 172.21.15.87:0/4009536959 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c02c7e0 con 0x7f044c003970 2024-01-20T08:30:55.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.470+0000 7f04675a9640 1 -- 172.21.15.87:0/4009536959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 msgr2=0x7f044c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.470+0000 7f04675a9640 1 --2- 172.21.15.87:0/4009536959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f044c003d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c110 tx=0x7f044c0057f0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.470+0000 7f0465b46640 1 -- 172.21.15.87:0/4009536959 reap_dead start 2024-01-20T08:30:55.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.470+0000 7f04675a9640 1 -- 172.21.15.87:0/4009536959 shutdown_connections 2024-01-20T08:30:55.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.470+0000 7f04675a9640 1 -- 172.21.15.87:0/4009536959 >> 172.21.15.87:0/4009536959 conn(0x7f046006b6c0 msgr2=0x7f046006baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:55.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.471+0000 7f04675a9640 1 -- 172.21.15.87:0/4009536959 shutdown_connections 2024-01-20T08:30:55.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.471+0000 7f04675a9640 1 -- 172.21.15.87:0/4009536959 wait complete. 2024-01-20T08:30:55.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.472+0000 7f04675a9640 1 Processor -- start 2024-01-20T08:30:55.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.473+0000 7f04675a9640 1 -- start start 2024-01-20T08:30:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f04675a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f04601efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f04675a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04600c2720 0x7f04601f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f04675a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 0x7f04600c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f04675a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04601e2e50 con 0x7f044c003970 2024-01-20T08:30:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f04675a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460153d80 con 0x7f04600c2720 2024-01-20T08:30:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f04675a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04600c1440 con 0x7f04600c84a0 2024-01-20T08:30:55.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.474+0000 7f0465b46640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 0x7f04600c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:55.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.475+0000 7f0464b44640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04600c2720 0x7f04601f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:55.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.475+0000 7f0465b46640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 0x7f04600c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50774/0 (socket says 172.21.15.87:50774) 2024-01-20T08:30:55.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.475+0000 7f0465b46640 1 -- 172.21.15.87:0/910552188 learned_addr learned my addr 172.21.15.87:0/910552188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:55.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.478+0000 7f0465b46640 1 -- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04600c2720 msgr2=0x7f04601f05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.478+0000 7f0465b46640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04600c2720 0x7f04601f05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.478+0000 7f0465b46640 1 -- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 msgr2=0x7f04601efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.478+0000 7f0465b46640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f04601efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.478+0000 7f0465b46640 1 -- 172.21.15.87:0/910552188 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04601f0110 con 0x7f04600c84a0 2024-01-20T08:30:55.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.481+0000 7f0465b46640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 0x7f04600c8920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f04600c3120 tx=0x7f045800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:55.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.485+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800a6d0 con 0x7f04600c84a0 2024-01-20T08:30:55.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.485+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045800a830 con 0x7f04600c84a0 2024-01-20T08:30:55.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.485+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800abe0 con 0x7f04600c84a0 2024-01-20T08:30:55.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.485+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04601542b0 con 0x7f04600c84a0 2024-01-20T08:30:55.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.485+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04600c0b10 con 0x7f04600c84a0 2024-01-20T08:30:55.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.486+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0460074290 con 0x7f04600c84a0 2024-01-20T08:30:55.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.504+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f045800ad40 con 0x7f04600c84a0 2024-01-20T08:30:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.505+0000 7f0455ffb640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f044406af00 0x7f044406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.505+0000 7f0465345640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f044406af00 0x7f044406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.506+0000 7f0465345640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f044406af00 0x7f044406d3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f045c034000 tx=0x7f045c02e000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.742+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0458058e70 con 0x7f04600c84a0 2024-01-20T08:30:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.754+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0458054ab0 con 0x7f04600c84a0 2024-01-20T08:30:55.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.974+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04600c1580 con 0x7f04600c84a0 2024-01-20T08:30:55.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.987+0000 7f0455ffb640 1 -- 172.21.15.87:0/910552188 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f04580580d0 con 0x7f04600c84a0 2024-01-20T08:30:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:30:55.990 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-20T08:30:55.990 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 1 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f044406af00 msgr2=0x7f044406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f044406af00 0x7f044406d3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f045c034000 tx=0x7f045c02e000 comp rx=0 tx=0).stop 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 msgr2=0x7f04600c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 0x7f04600c8920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f04600c3120 tx=0x7f045800cc20 comp rx=0 tx=0).stop 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 shutdown_connections 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f044406af00 0x7f044406d3c0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f045c034000 tx=0x7f045c02e000 comp rx=0 tx=0).stop 2024-01-20T08:30:55.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04600c84a0 0x7f04600c8920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04600c2720 0x7f04601f05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 --2- 172.21.15.87:0/910552188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f044c003970 0x7f04601efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:55.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 >> 172.21.15.87:0/910552188 conn(0x7f046006b6c0 msgr2=0x7f0460158160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:55.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 shutdown_connections 2024-01-20T08:30:55.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:55.989+0000 7f04675a9640 1 -- 172.21.15.87:0/910552188 wait complete. 2024-01-20T08:30:58.004 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:30:58.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.101+0000 7f7257326640 1 Processor -- start 2024-01-20T08:30:58.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.101+0000 7f7257326640 1 -- start start 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f7257326640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f725006a5b0 0x7f72500720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f7257326640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f72500c7870 0x7f725015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f7257326640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72500c7d00 con 0x7f7250153670 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f7257326640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7250153a50 con 0x7f725006a5b0 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f7257326640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7250072770 con 0x7f72500c7870 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f72548c1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f725006a5b0 0x7f72500720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.102+0000 7f72558c3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f72500c7870 0x7f725015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:58.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.103+0000 7f72548c1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f725006a5b0 0x7f72500720e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39752/0 (socket says 172.21.15.87:39752) 2024-01-20T08:30:58.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.103+0000 7f72548c1640 1 -- 172.21.15.87:0/4181028626 learned_addr learned my addr 172.21.15.87:0/4181028626 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.106+0000 7f72548c1640 1 -- 172.21.15.87:0/4181028626 >> v2:172.21.15.116:3300/0 conn(0x7f72500c7870 msgr2=0x7f725015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:30:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.106+0000 7f72548c1640 1 --2- 172.21.15.87:0/4181028626 >> v2:172.21.15.116:3300/0 conn(0x7f72500c7870 0x7f725015d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.106+0000 7f72548c1640 1 -- 172.21.15.87:0/4181028626 >> v1:172.21.15.87:6789/0 conn(0x7f7250153670 legacy=0x7f725015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.106+0000 7f72548c1640 1 -- 172.21.15.87:0/4181028626 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500c67d0 con 0x7f725006a5b0 2024-01-20T08:30:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.110+0000 7f72548c1640 1 --2- 172.21.15.87:0/4181028626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72500720e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7240009870 tx=0x7f724002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e738683fb3aa602a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.115+0000 7f7247fff640 1 -- 172.21.15.87:0/4181028626 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724002f850 con 0x7f725006a5b0 2024-01-20T08:30:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.115+0000 7f7247fff640 1 -- 172.21.15.87:0/4181028626 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724002f9b0 con 0x7f725006a5b0 2024-01-20T08:30:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.115+0000 7f7247fff640 1 -- 172.21.15.87:0/4181028626 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724002fd40 con 0x7f725006a5b0 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 -- 172.21.15.87:0/4181028626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 msgr2=0x7f72500720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 --2- 172.21.15.87:0/4181028626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72500720e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7240009870 tx=0x7f724002ba30 comp rx=0 tx=0).stop 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 -- 172.21.15.87:0/4181028626 shutdown_connections 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 --2- 172.21.15.87:0/4181028626 >> v2:172.21.15.116:3300/0 conn(0x7f72500c7870 0x7f725015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 --2- 172.21.15.87:0/4181028626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72500720e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 -- 172.21.15.87:0/4181028626 >> 172.21.15.87:0/4181028626 conn(0x7f72500cba70 msgr2=0x7f72500c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 -- 172.21.15.87:0/4181028626 shutdown_connections 2024-01-20T08:30:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.116+0000 7f7257326640 1 -- 172.21.15.87:0/4181028626 wait complete. 2024-01-20T08:30:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.117+0000 7f7257326640 1 Processor -- start 2024-01-20T08:30:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.117+0000 7f7257326640 1 -- start start 2024-01-20T08:30:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.118+0000 7f7257326640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72501f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.118+0000 7f7257326640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 0x7f7250150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.118+0000 7f7257326640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7250153670 0x7f725013ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.118+0000 7f7257326640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7250143ac0 con 0x7f725006a5b0 2024-01-20T08:30:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.118+0000 7f7257326640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501f01a0 con 0x7f72500c7870 2024-01-20T08:30:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.118+0000 7f7257326640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72501f3670 con 0x7f7250153670 2024-01-20T08:30:58.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.119+0000 7f72550c2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72501f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:58.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.119+0000 7f72548c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 0x7f7250150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:58.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.119+0000 7f72558c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7250153670 0x7f725013ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:58.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.119+0000 7f72548c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 0x7f7250150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:38668/0 (socket says 172.21.15.87:38668) 2024-01-20T08:30:58.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.119+0000 7f72548c1640 1 -- 172.21.15.87:0/1536724959 learned_addr learned my addr 172.21.15.87:0/1536724959 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:30:58.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.124+0000 7f72548c1640 1 -- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7250153670 msgr2=0x7f725013ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:58.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.124+0000 7f72548c1640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7250153670 0x7f725013ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.124+0000 7f72548c1640 1 -- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 msgr2=0x7f72501f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:58.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.124+0000 7f72548c1640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72501f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.124+0000 7f72548c1640 1 -- 172.21.15.87:0/1536724959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72400094d0 con 0x7f72500c7870 2024-01-20T08:30:58.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.128+0000 7f72548c1640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 0x7f7250150420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7240000c00 tx=0x7f72400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.133+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724003b040 con 0x7f72500c7870 2024-01-20T08:30:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.133+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240037c60 con 0x7f72500c7870 2024-01-20T08:30:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.133+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72501ef620 con 0x7f72500c7870 2024-01-20T08:30:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.133+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240042480 con 0x7f72500c7870 2024-01-20T08:30:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.133+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72501efbe0 con 0x7f72500c7870 2024-01-20T08:30:58.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.134+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72500c8a50 con 0x7f72500c7870 2024-01-20T08:30:58.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.169+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f72400425e0 con 0x7f72500c7870 2024-01-20T08:30:58.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.170+0000 7f7245ffb640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f723406af20 0x7f723406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:30:58.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.170+0000 7f72550c2640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f723406af20 0x7f723406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:30:58.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.170+0000 7f72550c2640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f723406af20 0x7f723406d3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f724c0016c0 tx=0x7f724c009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:30:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.174+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f724007be20 con 0x7f72500c7870 2024-01-20T08:30:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.457+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7240078160 con 0x7f72500c7870 2024-01-20T08:30:58.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.678+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7250074290 con 0x7f72500c7870 2024-01-20T08:30:58.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.801+0000 7f7245ffb640 1 -- 172.21.15.87:0/1536724959 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f724007b780 con 0x7f72500c7870 2024-01-20T08:30:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:30:58.805 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-20T08:30:58.805 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 1 2024-01-20T08:30:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f723406af20 msgr2=0x7f723406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f723406af20 0x7f723406d3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f724c0016c0 tx=0x7f724c009040 comp rx=0 tx=0).stop 2024-01-20T08:30:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 msgr2=0x7f7250150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:30:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 0x7f7250150420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7240000c00 tx=0x7f72400046c0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 shutdown_connections 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f723406af20 0x7f723406d3e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7250153670 0x7f725013ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72500c7870 0x7f7250150420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 --2- 172.21.15.87:0/1536724959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725006a5b0 0x7f72501f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 >> 172.21.15.87:0/1536724959 conn(0x7f72500cba70 msgr2=0x7f725014b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 shutdown_connections 2024-01-20T08:30:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:30:58.804+0000 7f7257326640 1 -- 172.21.15.87:0/1536724959 wait complete. 2024-01-20T08:31:00.166 INFO:tasks.ceph.mds.e.smithi087.stdout:starting mds.e at 2024-01-20T08:31:00.177 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-01-20T08:31:00.237 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-01-20T08:31:00.259 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-01-20T08:31:00.360 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-01-20T08:31:00.397 INFO:tasks.ceph.mds.f.smithi116.stdout:starting mds.f at 2024-01-20T08:31:00.819 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.919+0000 7fcf0029b640 1 Processor -- start 2024-01-20T08:31:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.919+0000 7fcf0029b640 1 -- start start 2024-01-20T08:31:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.920+0000 7fcf0029b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcef8153c20 0x7fcef8159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.920+0000 7fcf0029b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef815a740 con 0x7fcef80ca2b0 2024-01-20T08:31:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.920+0000 7fcf0029b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef806f770 con 0x7fcef80c7a20 2024-01-20T08:31:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.920+0000 7fcf0029b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef8159cc0 con 0x7fcef8153c20 2024-01-20T08:31:00.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.928+0000 7fcefe037640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcef8153c20 0x7fcef8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:00.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.929+0000 7fcefe037640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcef8153c20 0x7fcef8159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39830/0 (socket says 172.21.15.87:39830) 2024-01-20T08:31:00.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.929+0000 7fcefe037640 1 -- 172.21.15.87:0/1011521480 learned_addr learned my addr 172.21.15.87:0/1011521480 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:00.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.932+0000 7fcefe037640 1 -- 172.21.15.87:0/1011521480 >> v1:172.21.15.116:6789/0 conn(0x7fcef80c7a20 legacy=0x7fcef806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:00.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.932+0000 7fcefe037640 1 -- 172.21.15.87:0/1011521480 >> v1:172.21.15.87:6789/0 conn(0x7fcef80ca2b0 legacy=0x7fcef80c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:00.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.932+0000 7fcefe037640 1 -- 172.21.15.87:0/1011521480 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8159ea0 con 0x7fcef8153c20 2024-01-20T08:31:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.937+0000 7fcefe037640 1 --2- 172.21.15.87:0/1011521480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef8153c20 0x7fcef8159780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009870 tx=0x7fcef402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a8b2804a1bc4fa server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:00.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.953+0000 7fcefd035640 1 -- 172.21.15.87:0/1011521480 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4030860 con 0x7fcef8153c20 2024-01-20T08:31:00.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.953+0000 7fcefd035640 1 -- 172.21.15.87:0/1011521480 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef40309c0 con 0x7fcef8153c20 2024-01-20T08:31:00.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.953+0000 7fcefd035640 1 -- 172.21.15.87:0/1011521480 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4030d50 con 0x7fcef8153c20 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 -- 172.21.15.87:0/1011521480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef8153c20 msgr2=0x7fcef8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 --2- 172.21.15.87:0/1011521480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef8153c20 0x7fcef8159780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009870 tx=0x7fcef402b9a0 comp rx=0 tx=0).stop 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 -- 172.21.15.87:0/1011521480 shutdown_connections 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 --2- 172.21.15.87:0/1011521480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef8153c20 0x7fcef8159780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 -- 172.21.15.87:0/1011521480 >> 172.21.15.87:0/1011521480 conn(0x7fcef806a5b0 msgr2=0x7fcef806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 -- 172.21.15.87:0/1011521480 shutdown_connections 2024-01-20T08:31:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.954+0000 7fcf0029b640 1 -- 172.21.15.87:0/1011521480 wait complete. 2024-01-20T08:31:00.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.955+0000 7fcf0029b640 1 Processor -- start 2024-01-20T08:31:00.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.955+0000 7fcf0029b640 1 -- start start 2024-01-20T08:31:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcf0029b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcef80c7a20 0x7fcef8147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcf0029b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef80ca2b0 0x7fcef81efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcf0029b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 0x7fcef81577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcf0029b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80cb880 con 0x7fcef80ca2b0 2024-01-20T08:31:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcf0029b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef815ab50 con 0x7fcef8153c20 2024-01-20T08:31:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcf0029b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80cbdb0 con 0x7fcef80c7a20 2024-01-20T08:31:00.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcefe838640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 0x7fcef81577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:00.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.956+0000 7fcefe037640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcef80c7a20 0x7fcef8147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:00.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.957+0000 7fcefe838640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 0x7fcef81577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:38724/0 (socket says 172.21.15.87:38724) 2024-01-20T08:31:00.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.957+0000 7fcefe838640 1 -- 172.21.15.87:0/4614864 learned_addr learned my addr 172.21.15.87:0/4614864 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:00.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.960+0000 7fcefe838640 1 -- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcef80c7a20 msgr2=0x7fcef8147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:00.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.960+0000 7fcefe838640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcef80c7a20 0x7fcef8147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:00.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.960+0000 7fcefe838640 1 -- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef80ca2b0 msgr2=0x7fcef81efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:00.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.960+0000 7fcefe838640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef80ca2b0 0x7fcef81efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:00.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.960+0000 7fcefe838640 1 -- 172.21.15.87:0/4614864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40094d0 con 0x7fcef8153c20 2024-01-20T08:31:00.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.962+0000 7fcefe838640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 0x7fcef81577b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c7a0 tx=0x7fcef000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:00.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.966+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0003b70 con 0x7fcef8153c20 2024-01-20T08:31:00.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.966+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0003cd0 con 0x7fcef8153c20 2024-01-20T08:31:00.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.966+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef80cc270 con 0x7fcef8153c20 2024-01-20T08:31:00.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.966+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef000aa90 con 0x7fcef8153c20 2024-01-20T08:31:00.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.966+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81ef670 con 0x7fcef8153c20 2024-01-20T08:31:00.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:00.967+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef80c7ee0 con 0x7fcef8153c20 2024-01-20T08:31:01.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.256+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fcef000abf0 con 0x7fcef8153c20 2024-01-20T08:31:01.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.257+0000 7fceeeffd640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fcedc06af50 0x7fcedc06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:01.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.257+0000 7fcefe037640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fcedc06af50 0x7fcedc06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:01.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.257+0000 7fcefe037640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fcedc06af50 0x7fcedc06d410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcef4036000 tx=0x7fcef4008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.264+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcef0059100 con 0x7fcef8153c20 2024-01-20T08:31:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.274+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcef0054d40 con 0x7fcef8153c20 2024-01-20T08:31:01.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.484+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef8074290 con 0x7fcef8153c20 2024-01-20T08:31:01.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.494+0000 7fceeeffd640 1 -- 172.21.15.87:0/4614864 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7fcef0058360 con 0x7fcef8153c20 2024-01-20T08:31:01.497 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:01.497 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-20T08:31:01.497 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 2 2024-01-20T08:31:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fcedc06af50 msgr2=0x7fcedc06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fcedc06af50 0x7fcedc06d410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcef4036000 tx=0x7fcef4008030 comp rx=0 tx=0).stop 2024-01-20T08:31:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 msgr2=0x7fcef81577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 0x7fcef81577b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c7a0 tx=0x7fcef000cc60 comp rx=0 tx=0).stop 2024-01-20T08:31:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 shutdown_connections 2024-01-20T08:31:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fcedc06af50 0x7fcedc06d410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef8153c20 0x7fcef81577b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef80ca2b0 0x7fcef81efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 --2- 172.21.15.87:0/4614864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcef80c7a20 0x7fcef8147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 >> 172.21.15.87:0/4614864 conn(0x7fcef806a5b0 msgr2=0x7fcef80c8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 shutdown_connections 2024-01-20T08:31:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:01.496+0000 7fcf0029b640 1 -- 172.21.15.87:0/4614864 wait complete. 2024-01-20T08:31:03.510 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:03.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.607+0000 7fc508789640 1 Processor -- start 2024-01-20T08:31:03.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.607+0000 7fc508789640 1 -- start start 2024-01-20T08:31:03.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.608+0000 7fc508789640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5001518d0 0x7fc5000c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:03.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.608+0000 7fc508789640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000c88a0 con 0x7fc5001514f0 2024-01-20T08:31:03.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.608+0000 7fc508789640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000b1ad0 con 0x7fc50015b870 2024-01-20T08:31:03.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.608+0000 7fc508789640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000b2090 con 0x7fc5001518d0 2024-01-20T08:31:03.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.609+0000 7fc506525640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc5001514f0 0x7fc500158080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47354/0 (socket says 172.21.15.87:47354) 2024-01-20T08:31:03.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.609+0000 7fc506525640 1 -- 172.21.15.87:0/954644007 learned_addr learned my addr 172.21.15.87:0/954644007 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:03.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.609+0000 7fc505d24640 1 --2- 172.21.15.87:0/954644007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5001518d0 0x7fc5000c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:03.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.611+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325445356 0 0) 0x7fc5000c88a0 con 0x7fc5001514f0 2024-01-20T08:31:03.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.611+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4ec003520 con 0x7fc5001514f0 2024-01-20T08:31:03.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.612+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957169702 0 0) 0x7fc5000b1ad0 con 0x7fc50015b870 2024-01-20T08:31:03.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.612+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5000c88a0 con 0x7fc50015b870 2024-01-20T08:31:03.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.612+0000 7fc505d24640 1 -- 172.21.15.87:0/954644007 >> v1:172.21.15.116:6789/0 conn(0x7fc50015b870 legacy=0x7fc5000cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:03.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.612+0000 7fc505d24640 1 -- 172.21.15.87:0/954644007 >> v1:172.21.15.87:6789/0 conn(0x7fc5001514f0 legacy=0x7fc500158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:03.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.612+0000 7fc505d24640 1 -- 172.21.15.87:0/954644007 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000b1db0 con 0x7fc5001518d0 2024-01-20T08:31:03.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.614+0000 7fc505d24640 1 --2- 172.21.15.87:0/954644007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5001518d0 0x7fc5000c8360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00097d0 tx=0x7fc4f002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c604c1e1ae11fe server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:03.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.618+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0031890 con 0x7fc5001518d0 2024-01-20T08:31:03.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.618+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f00319f0 con 0x7fc5001518d0 2024-01-20T08:31:03.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.618+0000 7fc505523640 1 -- 172.21.15.87:0/954644007 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0031d80 con 0x7fc5001518d0 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.618+0000 7fc508789640 1 -- 172.21.15.87:0/954644007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5001518d0 msgr2=0x7fc5000c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.618+0000 7fc508789640 1 --2- 172.21.15.87:0/954644007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5001518d0 0x7fc5000c8360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00097d0 tx=0x7fc4f002b930 comp rx=0 tx=0).stop 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.619+0000 7fc508789640 1 -- 172.21.15.87:0/954644007 shutdown_connections 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.619+0000 7fc508789640 1 --2- 172.21.15.87:0/954644007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5001518d0 0x7fc5000c8360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.619+0000 7fc508789640 1 -- 172.21.15.87:0/954644007 >> 172.21.15.87:0/954644007 conn(0x7fc50006b6c0 msgr2=0x7fc50006baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.619+0000 7fc508789640 1 -- 172.21.15.87:0/954644007 shutdown_connections 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.619+0000 7fc508789640 1 -- 172.21.15.87:0/954644007 wait complete. 2024-01-20T08:31:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 Processor -- start 2024-01-20T08:31:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 -- start start 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5001514f0 0x7fc5001ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc50015b870 0x7fc50014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc500154560 0x7fc500152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001ed930 con 0x7fc5001514f0 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500151cc0 con 0x7fc50015b870 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.620+0000 7fc508789640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc500151fa0 con 0x7fc500154560 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.621+0000 7fc506525640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5001514f0 0x7fc5001ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.621+0000 7fc506d26640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc500154560 0x7fc500152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:03.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.621+0000 7fc505d24640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc50015b870 0x7fc50014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.621+0000 7fc506d26640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc500154560 0x7fc500152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50852/0 (socket says 172.21.15.87:50852) 2024-01-20T08:31:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.621+0000 7fc506d26640 1 -- 172.21.15.87:0/1552999679 learned_addr learned my addr 172.21.15.87:0/1552999679 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:03.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.624+0000 7fc506525640 1 -- 172.21.15.87:0/1552999679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc500154560 msgr2=0x7fc500152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:03.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.624+0000 7fc506525640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc500154560 0x7fc500152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:03.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.624+0000 7fc506525640 1 -- 172.21.15.87:0/1552999679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc50015b870 msgr2=0x7fc50014f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:03.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.624+0000 7fc506525640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc50015b870 0x7fc50014f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:03.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.624+0000 7fc506525640 1 -- 172.21.15.87:0/1552999679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0009400 con 0x7fc5001514f0 2024-01-20T08:31:03.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.627+0000 7fc506525640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5001514f0 0x7fc5001ed3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00cbb0 tx=0x7fc4fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:03.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.630+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00ac40 con 0x7fc5001514f0 2024-01-20T08:31:03.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.630+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc00ada0 con 0x7fc5001514f0 2024-01-20T08:31:03.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.630+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5000bfe90 con 0x7fc5001514f0 2024-01-20T08:31:03.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.630+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc012600 con 0x7fc5001514f0 2024-01-20T08:31:03.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.630+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc500153a30 con 0x7fc5001514f0 2024-01-20T08:31:03.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.631+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc500074290 con 0x7fc5001514f0 2024-01-20T08:31:03.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.649+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc4fc012760 con 0x7fc5001514f0 2024-01-20T08:31:03.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.650+0000 7fc4f6ffd640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc4e406af20 0x7fc4e406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:03.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.650+0000 7fc505d24640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc4e406af20 0x7fc4e406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:03.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.650+0000 7fc505d24640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc4e406af20 0x7fc4e406d3e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0036000 tx=0x7fc4f003c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.883+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4fc058990 con 0x7fc5001514f0 2024-01-20T08:31:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:03.883+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc4fc054cd0 con 0x7fc5001514f0 2024-01-20T08:31:04.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.089+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc500067240 con 0x7fc5001514f0 2024-01-20T08:31:04.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.120+0000 7fc4f6ffd640 1 -- 172.21.15.87:0/1552999679 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5277 (secure 0 0 0) 0x7fc4fc021090 con 0x7fc5001514f0 2024-01-20T08:31:04.123 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:04.123 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-20T08:31:04.124 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 3 2024-01-20T08:31:04.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc4e406af20 msgr2=0x7fc4e406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc4e406af20 0x7fc4e406d3e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0036000 tx=0x7fc4f003c040 comp rx=0 tx=0).stop 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5001514f0 msgr2=0x7fc5001ed3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5001514f0 0x7fc5001ed3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00cbb0 tx=0x7fc4fc00a060 comp rx=0 tx=0).stop 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 shutdown_connections 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc4e406af20 0x7fc4e406d3e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc500154560 0x7fc500152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc50015b870 0x7fc50014f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 --2- 172.21.15.87:0/1552999679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5001514f0 0x7fc5001ed3f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 >> 172.21.15.87:0/1552999679 conn(0x7fc50006b6c0 msgr2=0x7fc50014fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 shutdown_connections 2024-01-20T08:31:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.122+0000 7fc508789640 1 -- 172.21.15.87:0/1552999679 wait complete. 2024-01-20T08:31:04.137 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:04.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.235+0000 7f2b472af640 1 Processor -- start 2024-01-20T08:31:04.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.236+0000 7f2b472af640 1 -- start start 2024-01-20T08:31:04.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.236+0000 7f2b472af640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b40159e20 0x7f2b4006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.236+0000 7f2b472af640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4015d7b0 con 0x7f2b4006f580 2024-01-20T08:31:04.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.236+0000 7f2b472af640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b400cc440 con 0x7f2b401539a0 2024-01-20T08:31:04.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.236+0000 7f2b472af640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b4015dce0 con 0x7f2b40159e20 2024-01-20T08:31:04.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.243+0000 7f2b4484a640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2b4006f580 0x7f2b40159710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47362/0 (socket says 172.21.15.87:47362) 2024-01-20T08:31:04.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.243+0000 7f2b4484a640 1 -- 172.21.15.87:0/1923711701 learned_addr learned my addr 172.21.15.87:0/1923711701 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:04.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.246+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897709726 0 0) 0x7f2b4015d7b0 con 0x7f2b4006f580 2024-01-20T08:31:04.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.246+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2c003520 con 0x7f2b4006f580 2024-01-20T08:31:04.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.249+0000 7f2b4584c640 1 --2- 172.21.15.87:0/1923711701 >> v2:172.21.15.116:3300/0 conn(0x7f2b40159e20 0x7f2b4006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613159535 0 0) 0x7f2b2c003520 con 0x7f2b4006f580 2024-01-20T08:31:04.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 >> v2:172.21.15.116:3300/0 conn(0x7f2b40159e20 msgr2=0x7f2b4006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:04.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 --2- 172.21.15.87:0/1923711701 >> v2:172.21.15.116:3300/0 conn(0x7f2b40159e20 0x7f2b4006fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 >> v1:172.21.15.116:6789/0 conn(0x7f2b401539a0 legacy=0x7f2b400c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:04.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40070640 con 0x7f2b4006f580 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4017552009 0 0) 0x7f2b30001670 con 0x7f2b4006f580 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 >> v1:172.21.15.87:6789/0 conn(0x7f2b4006f580 legacy=0x7f2b40159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0039c0 0x7f2b2c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b2c0051d0 0x7f2b2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4015dce0 con 0x7f2b2c0039c0 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00c0e0 con 0x7f2b2c0051d0 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.250+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00c2a0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.251+0000 7f2b4484a640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.252+0000 7f2b4504b640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0039c0 0x7f2b2c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4584c640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b2c0051d0 0x7f2b2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4484a640 1 -- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b2c0051d0 msgr2=0x7f2b2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:04.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4484a640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b2c0051d0 0x7f2b2c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4484a640 1 -- 172.21.15.87:0/1923711701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0039c0 msgr2=0x7f2b2c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:04.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4484a640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0039c0 0x7f2b2c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4484a640 1 -- 172.21.15.87:0/1923711701 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c003520 con 0x7f2b2c0047c0 2024-01-20T08:31:04.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.253+0000 7f2b4584c640 1 -- 172.21.15.87:0/1923711701 reap_dead start 2024-01-20T08:31:04.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.255+0000 7f2b4484a640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c0e0 tx=0x7f2b2c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4cd15a1d595689 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:04.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.259+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30005ad0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.259+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b30005c30 con 0x7f2b2c0047c0 2024-01-20T08:31:04.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.259+0000 7f2b37fff640 1 -- 172.21.15.87:0/1923711701 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3002c9d0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 -- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b2c0047c0 msgr2=0x7f2b2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c0e0 tx=0x7f2b2c0057f0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 -- 172.21.15.87:0/1923711701 shutdown_connections 2024-01-20T08:31:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 --2- 172.21.15.87:0/1923711701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 -- 172.21.15.87:0/1923711701 >> 172.21.15.87:0/1923711701 conn(0x7f2b4006b6c0 msgr2=0x7f2b4006baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 -- 172.21.15.87:0/1923711701 shutdown_connections 2024-01-20T08:31:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.260+0000 7f2b472af640 1 -- 172.21.15.87:0/1923711701 wait complete. 2024-01-20T08:31:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.261+0000 7f2b472af640 1 Processor -- start 2024-01-20T08:31:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.261+0000 7f2b472af640 1 -- start start 2024-01-20T08:31:04.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b472af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0047c0 0x7f2b400c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b472af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b4006dd90 0x7f2b40154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b472af640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b40154ea0 0x7f2b401f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b472af640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c0940 con 0x7f2b2c0047c0 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b472af640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c0f60 con 0x7f2b4006dd90 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b472af640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c1b20 con 0x7f2b40154ea0 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b4484a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b4006dd90 0x7f2b40154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b4504b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0047c0 0x7f2b400c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.262+0000 7f2b4584c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b40154ea0 0x7f2b401f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.263+0000 7f2b4484a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b4006dd90 0x7f2b40154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:38774/0 (socket says 172.21.15.87:38774) 2024-01-20T08:31:04.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.263+0000 7f2b4484a640 1 -- 172.21.15.87:0/2289187325 learned_addr learned my addr 172.21.15.87:0/2289187325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:04.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.266+0000 7f2b4504b640 1 -- 172.21.15.87:0/2289187325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b40154ea0 msgr2=0x7f2b401f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.266+0000 7f2b4504b640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b40154ea0 0x7f2b401f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.266+0000 7f2b4504b640 1 -- 172.21.15.87:0/2289187325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b4006dd90 msgr2=0x7f2b40154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.266+0000 7f2b4504b640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b4006dd90 0x7f2b40154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.266+0000 7f2b4504b640 1 -- 172.21.15.87:0/2289187325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400c05f0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.268+0000 7f2b4504b640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0047c0 0x7f2b400c2810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00d4b0 tx=0x7f2b3c00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:04.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.272+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c012800 con 0x7f2b2c0047c0 2024-01-20T08:31:04.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.272+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c012960 con 0x7f2b2c0047c0 2024-01-20T08:31:04.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.272+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c012d10 con 0x7f2b2c0047c0 2024-01-20T08:31:04.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.272+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b400bfa20 con 0x7f2b2c0047c0 2024-01-20T08:31:04.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.272+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b400bfcc0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.273+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b40074290 con 0x7f2b2c0047c0 2024-01-20T08:31:04.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.291+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2b3c007020 con 0x7f2b2c0047c0 2024-01-20T08:31:04.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.292+0000 7f2b35ffb640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2b2406aed0 0x7f2b2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.292+0000 7f2b4484a640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2b2406aed0 0x7f2b2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.292+0000 7f2b4484a640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2b2406aed0 0x7f2b2406d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cbdd0 tx=0x7f2b30009030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.526+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b3c0585b0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.526+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2b3c0548f0 con 0x7f2b2c0047c0 2024-01-20T08:31:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.702+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b40070880 con 0x7f2b2c0047c0 2024-01-20T08:31:04.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.726+0000 7f2b35ffb640 1 -- 172.21.15.87:0/2289187325 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12851 (secure 0 0 0) 0x7f2b3c020020 con 0x7f2b2c0047c0 2024-01-20T08:31:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:04.729 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:30:47.867029+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2b2406aed0 msgr2=0x7f2b2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2b2406aed0 0x7f2b2406d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cbdd0 tx=0x7f2b30009030 comp rx=0 tx=0).stop 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0047c0 msgr2=0x7f2b400c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0047c0 0x7f2b400c2810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00d4b0 tx=0x7f2b3c00d970 comp rx=0 tx=0).stop 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 shutdown_connections 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2b2406aed0 0x7f2b2406d390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b40154ea0 0x7f2b401f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b4006dd90 0x7f2b40154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 --2- 172.21.15.87:0/2289187325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b2c0047c0 0x7f2b400c2810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 >> 172.21.15.87:0/2289187325 conn(0x7f2b4006b6c0 msgr2=0x7f2b4015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:04.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 shutdown_connections 2024-01-20T08:31:04.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.728+0000 7f2b472af640 1 -- 172.21.15.87:0/2289187325 wait complete. 2024-01-20T08:31:04.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-20T08:31:04.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:31:04.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.842+0000 7ff1a70c2640 1 Processor -- start 2024-01-20T08:31:04.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.842+0000 7ff1a70c2640 1 -- start start 2024-01-20T08:31:04.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.842+0000 7ff1a70c2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1a01517f0 0x7ff1a00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.842+0000 7ff1a70c2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a00ca070 con 0x7ff1a00c9550 2024-01-20T08:31:04.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.842+0000 7ff1a70c2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a0157ef0 con 0x7ff1a00c90e0 2024-01-20T08:31:04.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.842+0000 7ff1a70c2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a00b0520 con 0x7ff1a01517f0 2024-01-20T08:31:04.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.843+0000 7ff1a4e5e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1a01517f0 0x7ff1a00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.843+0000 7ff19ffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff1a00c90e0 0x7ff1a0157460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:42718/0 (socket says 172.21.15.87:42718) 2024-01-20T08:31:04.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.843+0000 7ff19ffff640 1 -- 172.21.15.87:0/3260398973 learned_addr learned my addr 172.21.15.87:0/3260398973 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:04.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.846+0000 7ff1a4e5e640 1 -- 172.21.15.87:0/3260398973 >> v1:172.21.15.116:6789/0 conn(0x7ff1a00c90e0 legacy=0x7ff1a0157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.846+0000 7ff1a4e5e640 1 -- 172.21.15.87:0/3260398973 >> v1:172.21.15.87:6789/0 conn(0x7ff1a00c9550 legacy=0x7ff1a00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.846+0000 7ff1a4e5e640 1 -- 172.21.15.87:0/3260398973 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a01581d0 con 0x7ff1a01517f0 2024-01-20T08:31:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.852+0000 7ff1a4e5e640 1 --2- 172.21.15.87:0/3260398973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a00b0dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097a0 tx=0x7ff19402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c2dc436d0d9fb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.855+0000 7ff19f7fe640 1 -- 172.21.15.87:0/3260398973 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194030610 con 0x7ff1a01517f0 2024-01-20T08:31:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff19f7fe640 1 -- 172.21.15.87:0/3260398973 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194030770 con 0x7ff1a01517f0 2024-01-20T08:31:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff19f7fe640 1 -- 172.21.15.87:0/3260398973 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194030b00 con 0x7ff1a01517f0 2024-01-20T08:31:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 -- 172.21.15.87:0/3260398973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 msgr2=0x7ff1a00b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/3260398973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a00b0dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097a0 tx=0x7ff19402b960 comp rx=0 tx=0).stop 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 -- 172.21.15.87:0/3260398973 shutdown_connections 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/3260398973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a00b0dd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 -- 172.21.15.87:0/3260398973 >> 172.21.15.87:0/3260398973 conn(0x7ff1a006a5b0 msgr2=0x7ff1a006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 -- 172.21.15.87:0/3260398973 shutdown_connections 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.856+0000 7ff1a70c2640 1 -- 172.21.15.87:0/3260398973 wait complete. 2024-01-20T08:31:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.858+0000 7ff1a70c2640 1 Processor -- start 2024-01-20T08:31:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.858+0000 7ff1a70c2640 1 -- start start 2024-01-20T08:31:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a70c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a00c90e0 0x7ff1a01edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a70c2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a00c9550 0x7ff1a01e4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a70c2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a01e1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a70c2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01e3130 con 0x7ff1a01517f0 2024-01-20T08:31:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a70c2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01ee510 con 0x7ff1a00c90e0 2024-01-20T08:31:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a70c2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01e0980 con 0x7ff1a00c9550 2024-01-20T08:31:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a565f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a01e1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a4e5e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a00c90e0 0x7ff1a01edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff19ffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a00c9550 0x7ff1a01e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a565f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a01e1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39886/0 (socket says 172.21.15.87:39886) 2024-01-20T08:31:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.859+0000 7ff1a565f640 1 -- 172.21.15.87:0/4246247436 learned_addr learned my addr 172.21.15.87:0/4246247436 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.863+0000 7ff1a565f640 1 -- 172.21.15.87:0/4246247436 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a00c9550 msgr2=0x7ff1a01e4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.863+0000 7ff1a565f640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a00c9550 0x7ff1a01e4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.863+0000 7ff1a565f640 1 -- 172.21.15.87:0/4246247436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a00c90e0 msgr2=0x7ff1a01edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.863+0000 7ff1a565f640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a00c90e0 0x7ff1a01edfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.863+0000 7ff1a565f640 1 -- 172.21.15.87:0/4246247436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194009400 con 0x7ff1a01517f0 2024-01-20T08:31:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.865+0000 7ff1a565f640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a01e1cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c7a0 tx=0x7ff19800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:04.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.868+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198003b70 con 0x7ff1a01517f0 2024-01-20T08:31:04.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.868+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198003cf0 con 0x7ff1a01517f0 2024-01-20T08:31:04.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.868+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198003ea0 con 0x7ff1a01517f0 2024-01-20T08:31:04.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.869+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a01e17a0 con 0x7ff1a01517f0 2024-01-20T08:31:04.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.869+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01e0f00 con 0x7ff1a01517f0 2024-01-20T08:31:04.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:04.869+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a01e1a10 con 0x7ff1a01517f0 2024-01-20T08:31:05.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.108+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff19800aa30 con 0x7ff1a01517f0 2024-01-20T08:31:05.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.108+0000 7ff19d7fa640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff18406aed0 0x7ff18406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:05.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.108+0000 7ff1a4e5e640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff18406aed0 0x7ff18406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:05.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.109+0000 7ff1a4e5e640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff18406aed0 0x7ff18406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff194036000 tx=0x7ff19403b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.121+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff1980589d0 con 0x7ff1a01517f0 2024-01-20T08:31:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.121+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff198054d10 con 0x7ff1a01517f0 2024-01-20T08:31:05.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.297+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1a0074290 con 0x7ff1a01517f0 2024-01-20T08:31:05.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.316+0000 7ff19d7fa640 1 -- 172.21.15.87:0/4246247436 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff198058330 con 0x7ff1a01517f0 2024-01-20T08:31:05.319 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.318+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff18406aed0 msgr2=0x7ff18406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.318+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff18406aed0 0x7ff18406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff194036000 tx=0x7ff19403b040 comp rx=0 tx=0).stop 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.318+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 msgr2=0x7ff1a01e1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.318+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a01e1cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c7a0 tx=0x7ff19800cc60 comp rx=0 tx=0).stop 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 shutdown_connections 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff18406aed0 0x7ff18406d390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a01517f0 0x7ff1a01e1cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a00c9550 0x7ff1a01e4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 --2- 172.21.15.87:0/4246247436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a00c90e0 0x7ff1a01edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 >> 172.21.15.87:0/4246247436 conn(0x7ff1a006a5b0 msgr2=0x7ff1a00c27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 shutdown_connections 2024-01-20T08:31:05.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.319+0000 7ff1a70c2640 1 -- 172.21.15.87:0/4246247436 wait complete. 2024-01-20T08:31:05.334 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-20T08:31:05.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.432+0000 7f54eb2b4640 1 Processor -- start 2024-01-20T08:31:05.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.433+0000 7f54eb2b4640 1 -- start start 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.433+0000 7f54eb2b4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54e4153980 0x7f54e40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.433+0000 7f54eb2b4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40c93c0 con 0x7f54e415d6a0 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.433+0000 7f54eb2b4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40ccfe0 con 0x7f54e40c9850 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.433+0000 7f54eb2b4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40c2e70 con 0x7f54e4153980 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.434+0000 7f54e884f640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f54e415d6a0 0x7f54e40c7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52750/0 (socket says 172.21.15.87:52750) 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.434+0000 7f54e884f640 1 -- 172.21.15.87:0/2262752401 learned_addr learned my addr 172.21.15.87:0/2262752401 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:05.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.434+0000 7f54e9050640 1 --2- 172.21.15.87:0/2262752401 >> v2:172.21.15.116:3300/0 conn(0x7f54e4153980 0x7f54e40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:05.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.436+0000 7f54dbfff640 1 -- 172.21.15.87:0/2262752401 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596050733 0 0) 0x7f54e40c93c0 con 0x7f54e415d6a0 2024-01-20T08:31:05.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.436+0000 7f54dbfff640 1 -- 172.21.15.87:0/2262752401 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54d0003520 con 0x7f54e415d6a0 2024-01-20T08:31:05.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.438+0000 7f54e9050640 1 -- 172.21.15.87:0/2262752401 >> v1:172.21.15.116:6789/0 conn(0x7f54e40c9850 legacy=0x7f54e40c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.438+0000 7f54e9050640 1 -- 172.21.15.87:0/2262752401 >> v1:172.21.15.87:6789/0 conn(0x7f54e415d6a0 legacy=0x7f54e40c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.438+0000 7f54e9050640 1 -- 172.21.15.87:0/2262752401 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40c3150 con 0x7f54e4153980 2024-01-20T08:31:05.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.440+0000 7f54e9050640 1 --2- 172.21.15.87:0/2262752401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e40cd350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009d90 tx=0x7f54dc02bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474401586659fcc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54dbfff640 1 -- 172.21.15.87:0/2262752401 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc02cd80 con 0x7f54e4153980 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54dbfff640 1 -- 172.21.15.87:0/2262752401 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc034440 con 0x7f54e4153980 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54dbfff640 1 -- 172.21.15.87:0/2262752401 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0347d0 con 0x7f54e4153980 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 -- 172.21.15.87:0/2262752401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 msgr2=0x7f54e40cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/2262752401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e40cd350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009d90 tx=0x7f54dc02bfd0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 -- 172.21.15.87:0/2262752401 shutdown_connections 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/2262752401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e40cd350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 -- 172.21.15.87:0/2262752401 >> 172.21.15.87:0/2262752401 conn(0x7f54e406a5b0 msgr2=0x7f54e406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 -- 172.21.15.87:0/2262752401 shutdown_connections 2024-01-20T08:31:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.446+0000 7f54eb2b4640 1 -- 172.21.15.87:0/2262752401 wait complete. 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 Processor -- start 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 -- start start 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54e40c9850 0x7f54e41f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e4150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54e415d6a0 0x7f54e41efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e415bdb0 con 0x7f54e415d6a0 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e414feb0 con 0x7f54e4153980 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.448+0000 7f54eb2b4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e4150100 con 0x7f54e40c9850 2024-01-20T08:31:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.449+0000 7f54e9851640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54e415d6a0 0x7f54e41efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.449+0000 7f54e884f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e4150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.449+0000 7f54e9050640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54e40c9850 0x7f54e41f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.449+0000 7f54e884f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e4150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:44046/0 (socket says 172.21.15.87:44046) 2024-01-20T08:31:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.449+0000 7f54e884f640 1 -- 172.21.15.87:0/3208296300 learned_addr learned my addr 172.21.15.87:0/3208296300 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.452+0000 7f54e9851640 1 -- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54e40c9850 msgr2=0x7f54e41f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.452+0000 7f54e9851640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54e40c9850 0x7f54e41f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.452+0000 7f54e9851640 1 -- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 msgr2=0x7f54e4150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.452+0000 7f54e9851640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e4150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.452+0000 7f54e9851640 1 -- 172.21.15.87:0/3208296300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009a70 con 0x7f54e415d6a0 2024-01-20T08:31:05.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.452+0000 7f54e884f640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e4150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:05.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.455+0000 7f54e9851640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54e415d6a0 0x7f54e41efbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c3c0 tx=0x7f54e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.458+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e00052d0 con 0x7f54e415d6a0 2024-01-20T08:31:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.458+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e0005430 con 0x7f54e415d6a0 2024-01-20T08:31:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.458+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0007260 con 0x7f54e415d6a0 2024-01-20T08:31:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.458+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e41503e0 con 0x7f54e415d6a0 2024-01-20T08:31:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.459+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e415abe0 con 0x7f54e415d6a0 2024-01-20T08:31:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.459+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e4074290 con 0x7f54e415d6a0 2024-01-20T08:31:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.478+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f54e00073c0 con 0x7f54e415d6a0 2024-01-20T08:31:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.478+0000 7f54d9ffb640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f54c806af00 0x7f54c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.479+0000 7f54e9050640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f54c806af00 0x7f54c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:05.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.479+0000 7f54e9050640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f54c806af00 0x7f54c806d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54dc036000 tx=0x7f54dc030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:05.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.713+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f54e0057cf0 con 0x7f54e415d6a0 2024-01-20T08:31:05.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.713+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f54e0054030 con 0x7f54e415d6a0 2024-01-20T08:31:05.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.884+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f54e414f0d0 con 0x7f54e415d6a0 2024-01-20T08:31:05.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.890+0000 7f54d9ffb640 1 -- 172.21.15.87:0/3208296300 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f54e0020070 con 0x7f54e415d6a0 2024-01-20T08:31:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:05.893 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-01-20T08:31:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-01-20T08:31:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-01-20T08:31:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-01-20T08:31:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-01-20T08:31:05.894 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f54c806af00 msgr2=0x7f54c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f54c806af00 0x7f54c806d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54dc036000 tx=0x7f54dc030000 comp rx=0 tx=0).stop 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54e415d6a0 msgr2=0x7f54e41efbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54e415d6a0 0x7f54e41efbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c3c0 tx=0x7f54e000c880 comp rx=0 tx=0).stop 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 shutdown_connections 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f54c806af00 0x7f54c806d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54e415d6a0 0x7f54e41efbd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54e4153980 0x7f54e4150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 --2- 172.21.15.87:0/3208296300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54e40c9850 0x7f54e41f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.892+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 >> 172.21.15.87:0/3208296300 conn(0x7f54e406a5b0 msgr2=0x7f54e406b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.893+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 shutdown_connections 2024-01-20T08:31:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:05.893+0000 7f54eb2b4640 1 -- 172.21.15.87:0/3208296300 wait complete. 2024-01-20T08:31:05.906 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-20T08:31:05.906 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-20T08:31:06.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.010+0000 7f6fd4e58640 1 Processor -- start 2024-01-20T08:31:06.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.010+0000 7f6fd4e58640 1 -- start start 2024-01-20T08:31:06.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.011+0000 7f6fd4e58640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6fd00c9e70 0x7f6fd0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:06.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.011+0000 7f6fd4e58640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6fd00b20e0 0x7f6fd0159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:06.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.011+0000 7f6fd4e58640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd015a3f0 con 0x7f6fd00c8fb0 2024-01-20T08:31:06.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.011+0000 7f6fd4e58640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd0154130 con 0x7f6fd00b20e0 2024-01-20T08:31:06.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.011+0000 7f6fd4e58640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00ca810 con 0x7f6fd00c9e70 2024-01-20T08:31:06.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.011+0000 7f6fced9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6fd00b20e0 0x7f6fd0159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:06.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.012+0000 7f6fced9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6fd00b20e0 0x7f6fd0159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56342/0 (socket says 172.21.15.87:56342) 2024-01-20T08:31:06.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.012+0000 7f6fced9d640 1 -- 172.21.15.87:0/4177388179 learned_addr learned my addr 172.21.15.87:0/4177388179 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:06.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.012+0000 7f6fcdd9b640 1 --2- 172.21.15.87:0/4177388179 >> v2:172.21.15.116:3300/0 conn(0x7f6fd00c9e70 0x7f6fd0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.014+0000 7f6fced9d640 1 -- 172.21.15.87:0/4177388179 >> v2:172.21.15.116:3300/0 conn(0x7f6fd00c9e70 msgr2=0x7f6fd0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.014+0000 7f6fced9d640 1 --2- 172.21.15.87:0/4177388179 >> v2:172.21.15.116:3300/0 conn(0x7f6fd00c9e70 0x7f6fd0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.014+0000 7f6fced9d640 1 -- 172.21.15.87:0/4177388179 >> v1:172.21.15.116:6789/0 conn(0x7f6fd00c8fb0 legacy=0x7f6fd0071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.014+0000 7f6fced9d640 1 -- 172.21.15.87:0/4177388179 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00c9390 con 0x7f6fd00b20e0 2024-01-20T08:31:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.016+0000 7f6fced9d640 1 --2- 172.21.15.87:0/4177388179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00b20e0 0x7f6fd0159eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0072920 tx=0x7f6fc802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a8aa3191d8dead server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:06.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fcd59a640 1 -- 172.21.15.87:0/4177388179 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc802cbb0 con 0x7f6fd00b20e0 2024-01-20T08:31:06.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fcd59a640 1 -- 172.21.15.87:0/4177388179 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc802cd10 con 0x7f6fd00b20e0 2024-01-20T08:31:06.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fcd59a640 1 -- 172.21.15.87:0/4177388179 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8034610 con 0x7f6fd00b20e0 2024-01-20T08:31:06.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fd4e58640 1 -- 172.21.15.87:0/4177388179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00b20e0 msgr2=0x7f6fd0159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:06.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/4177388179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00b20e0 0x7f6fd0159eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0072920 tx=0x7f6fc802c010 comp rx=0 tx=0).stop 2024-01-20T08:31:06.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fd4e58640 1 -- 172.21.15.87:0/4177388179 shutdown_connections 2024-01-20T08:31:06.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/4177388179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00b20e0 0x7f6fd0159eb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:06.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/4177388179 >> v2:172.21.15.116:3300/0 conn(0x7f6fd00c9e70 0x7f6fd0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:06.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.020+0000 7f6fd4e58640 1 -- 172.21.15.87:0/4177388179 >> 172.21.15.87:0/4177388179 conn(0x7f6fd008af90 msgr2=0x7f6fd008b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:06.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.021+0000 7f6fd4e58640 1 -- 172.21.15.87:0/4177388179 shutdown_connections 2024-01-20T08:31:06.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.021+0000 7f6fd4e58640 1 -- 172.21.15.87:0/4177388179 wait complete. 2024-01-20T08:31:06.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.022+0000 7f6fd4e58640 1 Processor -- start 2024-01-20T08:31:06.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.022+0000 7f6fd4e58640 1 -- start start 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fd4e58640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fd00b20e0 0x7f6fd01f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fd4e58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd00c8fb0 0x7f6fd014d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fd4e58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 0x7f6fd01527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fd4e58640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01561c0 con 0x7f6fd00c9e70 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fd4e58640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01ef280 con 0x7f6fd00c8fb0 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fd4e58640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01588a0 con 0x7f6fd00b20e0 2024-01-20T08:31:06.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.023+0000 7f6fced9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 0x7f6fd01527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:06.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.024+0000 7f6fcdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd00c8fb0 0x7f6fd014d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:06.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.024+0000 7f6fced9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 0x7f6fd01527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56346/0 (socket says 172.21.15.87:56346) 2024-01-20T08:31:06.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.024+0000 7f6fced9d640 1 -- 172.21.15.87:0/2049379785 learned_addr learned my addr 172.21.15.87:0/2049379785 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:06.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.024+0000 7f6fce59c640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fd00b20e0 0x7f6fd01f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:06.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.027+0000 7f6fced9d640 1 -- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fd00b20e0 msgr2=0x7f6fd01f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:06.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.027+0000 7f6fced9d640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fd00b20e0 0x7f6fd01f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:06.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.027+0000 7f6fced9d640 1 -- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd00c8fb0 msgr2=0x7f6fd014d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:06.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.027+0000 7f6fced9d640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd00c8fb0 0x7f6fd014d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:06.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.027+0000 7f6fced9d640 1 -- 172.21.15.87:0/2049379785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8009b30 con 0x7f6fd00c9e70 2024-01-20T08:31:06.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.029+0000 7f6fced9d640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 0x7f6fd01527a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009f40 tx=0x7f6fc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.033+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8009510 con 0x7f6fd00c9e70 2024-01-20T08:31:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.033+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc803e450 con 0x7f6fd00c9e70 2024-01-20T08:31:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.033+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc803e800 con 0x7f6fd00c9e70 2024-01-20T08:31:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.033+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd0151120 con 0x7f6fd00c9e70 2024-01-20T08:31:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.033+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd0158af0 con 0x7f6fd00c9e70 2024-01-20T08:31:06.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.034+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0159590 con 0x7f6fd00c9e70 2024-01-20T08:31:06.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.052+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6fc803e960 con 0x7f6fd00c9e70 2024-01-20T08:31:06.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.052+0000 7f6fbeffd640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6fb006af50 0x7f6fb006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:06.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.053+0000 7f6fce59c640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6fb006af50 0x7f6fb006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:06.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.053+0000 7f6fce59c640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6fb006af50 0x7f6fb006d410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0001780 tx=0x7f6fc0009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:06.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.285+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6fc807b4c0 con 0x7f6fd00c9e70 2024-01-20T08:31:06.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.285+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6fc8077800 con 0x7f6fd00c9e70 2024-01-20T08:31:06.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:06.468+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6fd01ef3f0 con 0x7f6fd00c9e70 2024-01-20T08:31:07.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.522+0000 7f6fbeffd640 1 -- 172.21.15.87:0/2049379785 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6fd01ef3f0 con 0x7f6fd00c9e70 2024-01-20T08:31:07.526 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6fb006af50 msgr2=0x7f6fb006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6fb006af50 0x7f6fb006d410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0001780 tx=0x7f6fc0009040 comp rx=0 tx=0).stop 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 msgr2=0x7f6fd01527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 0x7f6fd01527a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009f40 tx=0x7f6fc8004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 shutdown_connections 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6fb006af50 0x7f6fb006d410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fd00c9e70 0x7f6fd01527a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd00c8fb0 0x7f6fd014d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 --2- 172.21.15.87:0/2049379785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fd00b20e0 0x7f6fd01f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 >> 172.21.15.87:0/2049379785 conn(0x7f6fd008af90 msgr2=0x7f6fd0071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:07.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 shutdown_connections 2024-01-20T08:31:07.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.525+0000 7f6fd4e58640 1 -- 172.21.15.87:0/2049379785 wait complete. 2024-01-20T08:31:07.539 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-01-20T08:31:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.649+0000 7f3509303640 1 Processor -- start 2024-01-20T08:31:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.649+0000 7f3509303640 1 -- start start 2024-01-20T08:31:07.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.650+0000 7f3509303640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 0x7f35040ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.650+0000 7f3509303640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35040ca070 con 0x7f35041517f0 2024-01-20T08:31:07.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.650+0000 7f3509303640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3504157ef0 con 0x7f35040c90e0 2024-01-20T08:31:07.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.650+0000 7f3509303640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35040b0520 con 0x7f35040c9550 2024-01-20T08:31:07.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.689+0000 7f3502ffd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f35041517f0 0x7f35040b0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52764/0 (socket says 172.21.15.87:52764) 2024-01-20T08:31:07.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.689+0000 7f3502ffd640 1 -- 172.21.15.87:0/3493367432 learned_addr learned my addr 172.21.15.87:0/3493367432 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:07.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.690+0000 7f35037fe640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 0x7f35040ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.691+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165997079 0 0) 0x7f35040ca070 con 0x7f35041517f0 2024-01-20T08:31:07.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.691+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f0003520 con 0x7f35041517f0 2024-01-20T08:31:07.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2028026079 0 0) 0x7f34f0003520 con 0x7f35041517f0 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 msgr2=0x7f35040ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 0x7f35040ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 >> v1:172.21.15.116:6789/0 conn(0x7f35040c90e0 legacy=0x7f3504157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35041581d0 con 0x7f35041517f0 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4017552009 0 0) 0x7f34f4001670 con 0x7f35041517f0 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 >> v1:172.21.15.87:6789/0 conn(0x7f35041517f0 legacy=0x7f35040b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 0x7f34f0003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34f0004150 0x7f34f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34f0004e20 0x7f34f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35040b0520 con 0x7f34f0004e20 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00055a0 con 0x7f34f0004150 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.695+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f000bec0 con 0x7f35040c9550 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.697+0000 7f35037fe640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34f0004e20 0x7f34f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.698+0000 7f3502ffd640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 0x7f34f0003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.700+0000 7f35037fe640 1 -- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 msgr2=0x7f34f0003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.700+0000 7f35037fe640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35040c9550 0x7f34f0003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.700+0000 7f35037fe640 1 -- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34f0004150 msgr2=0x7f34f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.700+0000 7f35037fe640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34f0004150 0x7f34f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.700+0000 7f35037fe640 1 -- 172.21.15.87:0/3493367432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f0003520 con 0x7f34f0004e20 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.703+0000 7f35037fe640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34f0004e20 0x7f34f000b980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007f60 tx=0x7f34f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe3abd1b8bf96d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.721+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc033600 con 0x7f34f0004e20 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.721+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc02f050 con 0x7f34f0004e20 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.721+0000 7f35027fc640 1 -- 172.21.15.87:0/3493367432 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0339e0 con 0x7f34f0004e20 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f3509303640 1 -- 172.21.15.87:0/3493367432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34f0004e20 msgr2=0x7f34f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f3509303640 1 --2- 172.21.15.87:0/3493367432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34f0004e20 0x7f34f000b980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007f60 tx=0x7f34f00057f0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f35037fe640 1 -- 172.21.15.87:0/3493367432 reap_dead start 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f3509303640 1 -- 172.21.15.87:0/3493367432 shutdown_connections 2024-01-20T08:31:07.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f3509303640 1 -- 172.21.15.87:0/3493367432 >> 172.21.15.87:0/3493367432 conn(0x7f350406a5b0 msgr2=0x7f350406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f3509303640 1 -- 172.21.15.87:0/3493367432 shutdown_connections 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.722+0000 7f3509303640 1 -- 172.21.15.87:0/3493367432 wait complete. 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.724+0000 7f3509303640 1 Processor -- start 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.724+0000 7f3509303640 1 -- start start 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3509303640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 0x7f35041ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3509303640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350414f160 0x7f3504146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3509303640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f350414c180 0x7f3504146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3509303640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35040ad460 con 0x7f350414c180 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3509303640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350414e8d0 con 0x7f350414f160 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3509303640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350406b9f0 con 0x7f350414bda0 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f35037fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f350414c180 0x7f3504146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 0x7f35041ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f34fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350414f160 0x7f3504146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 0x7f35041ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49788/0 (socket says 172.21.15.87:49788) 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.725+0000 7f3502ffd640 1 -- 172.21.15.87:0/2192098392 learned_addr learned my addr 172.21.15.87:0/2192098392 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.728+0000 7f3502ffd640 1 -- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350414f160 msgr2=0x7f3504146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.728+0000 7f3502ffd640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350414f160 0x7f3504146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.728+0000 7f3502ffd640 1 -- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f350414c180 msgr2=0x7f3504146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.728+0000 7f3502ffd640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f350414c180 0x7f3504146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.728+0000 7f3502ffd640 1 -- 172.21.15.87:0/2192098392 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350406c290 con 0x7f350414bda0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.731+0000 7f3502ffd640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 0x7f35041ef030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34f4001a90 tx=0x7f34f4007720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.732+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400f940 con 0x7f350414bda0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.733+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3504147180 con 0x7f350414bda0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.733+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350414eae0 con 0x7f350414bda0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.733+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400faa0 con 0x7f350414bda0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.734+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400fe10 con 0x7f350414bda0 2024-01-20T08:31:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.734+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35040cabe0 con 0x7f350414bda0 2024-01-20T08:31:07.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.973+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f34f401f440 con 0x7f350414bda0 2024-01-20T08:31:07.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.973+0000 7f34fb7fe640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f34e406af00 0x7f34e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:07.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.973+0000 7f34fbfff640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f34e406af00 0x7f34e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:07.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.974+0000 7f34fbfff640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f34e406af00 0x7f34e406d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35040c8c60 tx=0x7f34ec007db0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:07.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.976+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f34f4058400 con 0x7f350414bda0 2024-01-20T08:31:07.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:07.988+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f34f4054740 con 0x7f350414bda0 2024-01-20T08:31:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.160+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f3504074290 con 0x7f350414bda0 2024-01-20T08:31:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.634+0000 7f34fb7fe640 1 -- 172.21.15.87:0/2192098392 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f34f401a020 con 0x7f350414bda0 2024-01-20T08:31:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-01-20T08:31:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f34e406af00 msgr2=0x7f34e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f34e406af00 0x7f34e406d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35040c8c60 tx=0x7f34ec007db0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 msgr2=0x7f35041ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 0x7f35041ef030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34f4001a90 tx=0x7f34f4007720 comp rx=0 tx=0).stop 2024-01-20T08:31:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 shutdown_connections 2024-01-20T08:31:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f34e406af00 0x7f34e406d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f350414c180 0x7f3504146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350414f160 0x7f3504146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 --2- 172.21.15.87:0/2192098392 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350414bda0 0x7f35041ef030 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 >> 172.21.15.87:0/2192098392 conn(0x7f350406a5b0 msgr2=0x7f35040cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 shutdown_connections 2024-01-20T08:31:08.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.636+0000 7f3509303640 1 -- 172.21.15.87:0/2192098392 wait complete. 2024-01-20T08:31:08.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-20T08:31:08.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.755+0000 7fbf36243640 1 Processor -- start 2024-01-20T08:31:08.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.755+0000 7fbf36243640 1 -- start start 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.756+0000 7fbf36243640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbf301502e0 0x7fbf300c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.756+0000 7fbf36243640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf300c6760 0x7fbf300c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.756+0000 7fbf36243640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300c84b0 con 0x7fbf300c8080 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.756+0000 7fbf36243640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3006ea70 con 0x7fbf301502e0 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.756+0000 7fbf36243640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3006ed10 con 0x7fbf300c6760 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.757+0000 7fbf2effd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf300c6760 0x7fbf300c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.757+0000 7fbf2effd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf300c6760 0x7fbf300c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:44084/0 (socket says 172.21.15.87:44084) 2024-01-20T08:31:08.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.757+0000 7fbf2effd640 1 -- 172.21.15.87:0/4233877594 learned_addr learned my addr 172.21.15.87:0/4233877594 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:08.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.759+0000 7fbf2effd640 1 -- 172.21.15.87:0/4233877594 >> v2:172.21.15.87:3300/0 conn(0x7fbf301502e0 msgr2=0x7fbf300c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:08.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.759+0000 7fbf2effd640 1 --2- 172.21.15.87:0/4233877594 >> v2:172.21.15.87:3300/0 conn(0x7fbf301502e0 0x7fbf300c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.759+0000 7fbf2effd640 1 -- 172.21.15.87:0/4233877594 >> v1:172.21.15.87:6789/0 conn(0x7fbf300c8080 legacy=0x7fbf300c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:08.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.759+0000 7fbf2effd640 1 -- 172.21.15.87:0/4233877594 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3006dd90 con 0x7fbf300c6760 2024-01-20T08:31:08.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.761+0000 7fbf2effd640 1 --2- 172.21.15.87:0/4233877594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf300c2b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf300c3130 tx=0x7fbf2402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a75fe326732b25 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:08.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.765+0000 7fbf2e7fc640 1 -- 172.21.15.87:0/4233877594 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24030860 con 0x7fbf300c6760 2024-01-20T08:31:08.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.765+0000 7fbf2e7fc640 1 -- 172.21.15.87:0/4233877594 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf240309c0 con 0x7fbf300c6760 2024-01-20T08:31:08.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.765+0000 7fbf2e7fc640 1 -- 172.21.15.87:0/4233877594 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24030d50 con 0x7fbf300c6760 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 -- 172.21.15.87:0/4233877594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 msgr2=0x7fbf300c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 --2- 172.21.15.87:0/4233877594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf300c2b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf300c3130 tx=0x7fbf2402b9a0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 -- 172.21.15.87:0/4233877594 shutdown_connections 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 --2- 172.21.15.87:0/4233877594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf300c2b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 --2- 172.21.15.87:0/4233877594 >> v2:172.21.15.87:3300/0 conn(0x7fbf301502e0 0x7fbf300c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 -- 172.21.15.87:0/4233877594 >> 172.21.15.87:0/4233877594 conn(0x7fbf300723c0 msgr2=0x7fbf3006fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:08.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 -- 172.21.15.87:0/4233877594 shutdown_connections 2024-01-20T08:31:08.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.766+0000 7fbf36243640 1 -- 172.21.15.87:0/4233877594 wait complete. 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.767+0000 7fbf36243640 1 Processor -- start 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.767+0000 7fbf36243640 1 -- start start 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf36243640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf3006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf36243640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf300c8080 0x7fbf30149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf36243640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf301502e0 0x7fbf30157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf36243640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30158dc0 con 0x7fbf301502e0 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf36243640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301ecf80 con 0x7fbf300c6760 2024-01-20T08:31:08.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf36243640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3006c560 con 0x7fbf300c8080 2024-01-20T08:31:08.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf3006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:08.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.768+0000 7fbf2effd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf300c8080 0x7fbf30149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:08.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.769+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf3006a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:44086/0 (socket says 172.21.15.87:44086) 2024-01-20T08:31:08.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.769+0000 7fbf2f7fe640 1 -- 172.21.15.87:0/1033880693 learned_addr learned my addr 172.21.15.87:0/1033880693 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.773+0000 7fbf2f7fe640 1 -- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf300c8080 msgr2=0x7fbf30149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.773+0000 7fbf2f7fe640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf300c8080 0x7fbf30149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.773+0000 7fbf2f7fe640 1 -- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf301502e0 msgr2=0x7fbf30157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.773+0000 7fbf2f7fe640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf301502e0 0x7fbf30157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.773+0000 7fbf2f7fe640 1 -- 172.21.15.87:0/1033880693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf240094d0 con 0x7fbf300c6760 2024-01-20T08:31:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.776+0000 7fbf2f7fe640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf3006a5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b440 tx=0x7fbf2000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:08.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.779+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20011890 con 0x7fbf300c6760 2024-01-20T08:31:08.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.779+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf200119f0 con 0x7fbf300c6760 2024-01-20T08:31:08.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.779+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf301edf50 con 0x7fbf300c6760 2024-01-20T08:31:08.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.779+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20011da0 con 0x7fbf300c6760 2024-01-20T08:31:08.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.780+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf301edac0 con 0x7fbf300c6760 2024-01-20T08:31:08.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:08.780+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf30156ec0 con 0x7fbf300c6760 2024-01-20T08:31:09.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.060+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbf2001a400 con 0x7fbf300c6760 2024-01-20T08:31:09.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.061+0000 7fbf17fff640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbf1c06af50 0x7fbf1c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:09.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.061+0000 7fbf2effd640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbf1c06af50 0x7fbf1c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:09.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.061+0000 7fbf2effd640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbf1c06af50 0x7fbf1c06d410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf24036000 tx=0x7fbf24008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:09.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.063+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbf20059140 con 0x7fbf300c6760 2024-01-20T08:31:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.077+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbf20054d80 con 0x7fbf300c6760 2024-01-20T08:31:09.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:09.293+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbf30074290 con 0x7fbf300c6760 2024-01-20T08:31:09.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:31:09.360 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:31:10.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.630+0000 7fbf17fff640 1 -- 172.21.15.87:0/1033880693 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbf20015020 con 0x7fbf300c6760 2024-01-20T08:31:10.633 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 2 and data pool 3 2024-01-20T08:31:10.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbf1c06af50 msgr2=0x7fbf1c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbf1c06af50 0x7fbf1c06d410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf24036000 tx=0x7fbf24008030 comp rx=0 tx=0).stop 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 msgr2=0x7fbf3006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf3006a5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b440 tx=0x7fbf2000b900 comp rx=0 tx=0).stop 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 shutdown_connections 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbf1c06af50 0x7fbf1c06d410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf301502e0 0x7fbf30157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf300c8080 0x7fbf30149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 --2- 172.21.15.87:0/1033880693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300c6760 0x7fbf3006a5b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 >> 172.21.15.87:0/1033880693 conn(0x7fbf300723c0 msgr2=0x7fbf300c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 shutdown_connections 2024-01-20T08:31:10.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.632+0000 7fbf36243640 1 -- 172.21.15.87:0/1033880693 wait complete. 2024-01-20T08:31:10.645 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-01-20T08:31:10.645 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-01-20T08:31:10.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.747+0000 7f060d782640 1 Processor -- start 2024-01-20T08:31:10.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.747+0000 7f060d782640 1 -- start start 2024-01-20T08:31:10.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.748+0000 7f060d782640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06080727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:10.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.748+0000 7f060d782640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0608156070 con 0x7f06081590e0 2024-01-20T08:31:10.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.748+0000 7f060d782640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06081501c0 con 0x7f06080721b0 2024-01-20T08:31:10.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.748+0000 7f060d782640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0608155af0 con 0x7f060814fde0 2024-01-20T08:31:10.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.749+0000 7f06077fe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f06081590e0 0x7f0608071120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52782/0 (socket says 172.21.15.87:52782) 2024-01-20T08:31:10.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.749+0000 7f06077fe640 1 -- 172.21.15.87:0/807432072 learned_addr learned my addr 172.21.15.87:0/807432072 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:10.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.749+0000 7f0606ffd640 1 --2- 172.21.15.87:0/807432072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06080727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157984748 0 0) 0x7f06081501c0 con 0x7f06080721b0 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f0004630 con 0x7f06080721b0 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712441476 0 0) 0x7f0608156070 con 0x7f06081590e0 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06081501c0 con 0x7f06081590e0 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0606ffd640 1 -- 172.21.15.87:0/807432072 >> v1:172.21.15.116:6789/0 conn(0x7f06080721b0 legacy=0x7f0608156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0606ffd640 1 -- 172.21.15.87:0/807432072 >> v1:172.21.15.87:6789/0 conn(0x7f06081590e0 legacy=0x7f0608071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.751+0000 7f0606ffd640 1 -- 172.21.15.87:0/807432072 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060806edb0 con 0x7f060814fde0 2024-01-20T08:31:10.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.754+0000 7f0606ffd640 1 --2- 172.21.15.87:0/807432072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06080727c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0600009dc0 tx=0x7f060002bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5146f968d05b00f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:10.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.757+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060002cd50 con 0x7f060814fde0 2024-01-20T08:31:10.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.757+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600034440 con 0x7f060814fde0 2024-01-20T08:31:10.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.757+0000 7f0605ffb640 1 -- 172.21.15.87:0/807432072 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06000347d0 con 0x7f060814fde0 2024-01-20T08:31:10.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 -- 172.21.15.87:0/807432072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 msgr2=0x7f06080727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 --2- 172.21.15.87:0/807432072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06080727c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0600009dc0 tx=0x7f060002bfa0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 -- 172.21.15.87:0/807432072 shutdown_connections 2024-01-20T08:31:10.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 --2- 172.21.15.87:0/807432072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06080727c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 -- 172.21.15.87:0/807432072 >> 172.21.15.87:0/807432072 conn(0x7f060806d510 msgr2=0x7f06080663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:10.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 -- 172.21.15.87:0/807432072 shutdown_connections 2024-01-20T08:31:10.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.758+0000 7f060d782640 1 -- 172.21.15.87:0/807432072 wait complete. 2024-01-20T08:31:10.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.760+0000 7f060d782640 1 Processor -- start 2024-01-20T08:31:10.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.760+0000 7f060d782640 1 -- start start 2024-01-20T08:31:10.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.761+0000 7f060d782640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 0x7f06080c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:10.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.761+0000 7f060d782640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06081ed430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:10.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.761+0000 7f060d782640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06081590e0 0x7f06081ed970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:10.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.761+0000 7f060d782640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06080c5180 con 0x7f06080721b0 2024-01-20T08:31:10.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.761+0000 7f060d782640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06081ecf70 con 0x7f06081590e0 2024-01-20T08:31:10.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.761+0000 7f060d782640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06080caaf0 con 0x7f060814fde0 2024-01-20T08:31:10.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.762+0000 7f06067fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06081ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:10.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.762+0000 7f06077fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06081590e0 0x7f06081ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:10.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.762+0000 7f0606ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 0x7f06080c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:10.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.762+0000 7f0606ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 0x7f06080c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56412/0 (socket says 172.21.15.87:56412) 2024-01-20T08:31:10.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.762+0000 7f0606ffd640 1 -- 172.21.15.87:0/2439834896 learned_addr learned my addr 172.21.15.87:0/2439834896 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:10.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.765+0000 7f0606ffd640 1 -- 172.21.15.87:0/2439834896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 msgr2=0x7f06081ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.765+0000 7f0606ffd640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06081ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.765+0000 7f0606ffd640 1 -- 172.21.15.87:0/2439834896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06081590e0 msgr2=0x7f06081ed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:10.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.765+0000 7f0606ffd640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06081590e0 0x7f06081ed970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:10.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.765+0000 7f0606ffd640 1 -- 172.21.15.87:0/2439834896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0600009a70 con 0x7f06080721b0 2024-01-20T08:31:10.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.768+0000 7f0606ffd640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 0x7f06080c6d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0600000f80 tx=0x7f0600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:10.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.771+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060003e410 con 0x7f06080721b0 2024-01-20T08:31:10.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.771+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060003e570 con 0x7f06080721b0 2024-01-20T08:31:10.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.771+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06080c4bc0 con 0x7f06080721b0 2024-01-20T08:31:10.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.771+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060003e920 con 0x7f06080721b0 2024-01-20T08:31:10.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.771+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06080c2500 con 0x7f06080721b0 2024-01-20T08:31:10.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.773+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0608074290 con 0x7f06080721b0 2024-01-20T08:31:10.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.793+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f060003b050 con 0x7f06080721b0 2024-01-20T08:31:10.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.793+0000 7f05ef7fe640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f05f406af50 0x7f05f406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:10.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.793+0000 7f06067fc640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f05f406af50 0x7f05f406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:10.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:10.794+0000 7f06067fc640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f05f406af50 0x7f05f406d410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06080ca660 tx=0x7f05fc009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:11.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:11.030+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f060007ac20 con 0x7f06080721b0 2024-01-20T08:31:11.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:11.030+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0600076f60 con 0x7f06080721b0 2024-01-20T08:31:11.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:11.232+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0608071c60 con 0x7f06080721b0 2024-01-20T08:31:12.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.259+0000 7f05ef7fe640 1 -- 172.21.15.87:0/2439834896 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f06000390e0 con 0x7f06080721b0 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f05f406af50 msgr2=0x7f05f406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f05f406af50 0x7f05f406d410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06080ca660 tx=0x7f05fc009040 comp rx=0 tx=0).stop 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 msgr2=0x7f06080c6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 0x7f06080c6d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0600000f80 tx=0x7f0600004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 shutdown_connections 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f05f406af50 0x7f05f406d410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06081590e0 0x7f06081ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f060814fde0 0x7f06081ed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 --2- 172.21.15.87:0/2439834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06080721b0 0x7f06080c6d30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 >> 172.21.15.87:0/2439834896 conn(0x7f060806d510 msgr2=0x7f0608156500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:12.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 shutdown_connections 2024-01-20T08:31:12.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.262+0000 7f060d782640 1 -- 172.21.15.87:0/2439834896 wait complete. 2024-01-20T08:31:12.276 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-01-20T08:31:12.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.386+0000 7fd9adbea640 1 Processor -- start 2024-01-20T08:31:12.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.386+0000 7fd9adbea640 1 -- start start 2024-01-20T08:31:12.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9adbea640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a8157920 0x7fd9a815a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:12.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9adbea640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a80c0f20 con 0x7fd9a81506f0 2024-01-20T08:31:12.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9adbea640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a8156da0 con 0x7fd9a8156430 2024-01-20T08:31:12.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9adbea640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c91c0 con 0x7fd9a8157920 2024-01-20T08:31:12.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a8157920 0x7fd9a815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:12.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9a7fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd9a8156430 0x7fd9a806fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:50682/0 (socket says 172.21.15.87:50682) 2024-01-20T08:31:12.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.387+0000 7fd9a7fff640 1 -- 172.21.15.87:0/2152314976 learned_addr learned my addr 172.21.15.87:0/2152314976 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:12.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.390+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/2152314976 >> v1:172.21.15.116:6789/0 conn(0x7fd9a8156430 legacy=0x7fd9a806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:12.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.390+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/2152314976 >> v1:172.21.15.87:6789/0 conn(0x7fd9a81506f0 legacy=0x7fd9a80c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:12.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.390+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/2152314976 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8157060 con 0x7fd9a8157920 2024-01-20T08:31:12.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.395+0000 7fd9a6ffd640 1 --2- 172.21.15.87:0/2152314976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a8157920 0x7fd9a815a450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097d0 tx=0x7fd99c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50942d8d1e25ab02 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.474+0000 7fd9a67fc640 1 -- 172.21.15.87:0/2152314976 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c031770 con 0x7fd9a8157920 2024-01-20T08:31:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.474+0000 7fd9a67fc640 1 -- 172.21.15.87:0/2152314976 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c0318d0 con 0x7fd9a8157920 2024-01-20T08:31:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.474+0000 7fd9a67fc640 1 -- 172.21.15.87:0/2152314976 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c031c60 con 0x7fd9a8157920 2024-01-20T08:31:12.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 -- 172.21.15.87:0/2152314976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a8157920 msgr2=0x7fd9a815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:12.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 --2- 172.21.15.87:0/2152314976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a8157920 0x7fd9a815a450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097d0 tx=0x7fd99c02b810 comp rx=0 tx=0).stop 2024-01-20T08:31:12.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 -- 172.21.15.87:0/2152314976 shutdown_connections 2024-01-20T08:31:12.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 --2- 172.21.15.87:0/2152314976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a8157920 0x7fd9a815a450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 -- 172.21.15.87:0/2152314976 >> 172.21.15.87:0/2152314976 conn(0x7fd9a8072810 msgr2=0x7fd9a806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 -- 172.21.15.87:0/2152314976 shutdown_connections 2024-01-20T08:31:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.475+0000 7fd9adbea640 1 -- 172.21.15.87:0/2152314976 wait complete. 2024-01-20T08:31:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.476+0000 7fd9adbea640 1 Processor -- start 2024-01-20T08:31:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 -- start start 2024-01-20T08:31:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a81506f0 0x7fd9a80c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 0x7fd9a8147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8157920 0x7fd9a80cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c9a10 con 0x7fd9a8157920 2024-01-20T08:31:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c9c60 con 0x7fd9a8156430 2024-01-20T08:31:12.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.477+0000 7fd9adbea640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c9f40 con 0x7fd9a81506f0 2024-01-20T08:31:12.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.479+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 0x7fd9a8147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.479+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 0x7fd9a8147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:44112/0 (socket says 172.21.15.87:44112) 2024-01-20T08:31:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.479+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/1023677375 learned_addr learned my addr 172.21.15.87:0/1023677375 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.481+0000 7fd9a7fff640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8157920 0x7fd9a80cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:12.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.481+0000 7fd9a77fe640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a81506f0 0x7fd9a80c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.483+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a81506f0 msgr2=0x7fd9a80c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.483+0000 7fd9a6ffd640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a81506f0 0x7fd9a80c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.483+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8157920 msgr2=0x7fd9a80cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.483+0000 7fd9a6ffd640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8157920 0x7fd9a80cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.483+0000 7fd9a6ffd640 1 -- 172.21.15.87:0/1023677375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009400 con 0x7fd9a8156430 2024-01-20T08:31:12.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.485+0000 7fd9a6ffd640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 0x7fd9a8147c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097d0 tx=0x7fd99c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.488+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c03c950 con 0x7fd9a8156430 2024-01-20T08:31:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.488+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c03cab0 con 0x7fd9a8156430 2024-01-20T08:31:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.488+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c03ce60 con 0x7fd9a8156430 2024-01-20T08:31:12.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.488+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a80c6d40 con 0x7fd9a8156430 2024-01-20T08:31:12.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.489+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80c83b0 con 0x7fd9a8156430 2024-01-20T08:31:12.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.490+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a80c7260 con 0x7fd9a8156430 2024-01-20T08:31:12.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.748+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd99c04f450 con 0x7fd9a8156430 2024-01-20T08:31:12.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.748+0000 7fd98ffff640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd99406af50 0x7fd99406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:12.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.748+0000 7fd9a77fe640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd99406af50 0x7fd99406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:12.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.748+0000 7fd9a77fe640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd99406af50 0x7fd99406d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd998006fd0 tx=0x7fd998009060 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:12.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.751+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fd99c07ab80 con 0x7fd9a8156430 2024-01-20T08:31:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.761+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd99c0767c0 con 0x7fd9a8156430 2024-01-20T08:31:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:12.943+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd9a80c7260 con 0x7fd9a8156430 2024-01-20T08:31:15.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.009+0000 7fd98ffff640 1 -- 172.21.15.87:0/1023677375 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fd9a80c7260 con 0x7fd9a8156430 2024-01-20T08:31:15.013 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data_ec' created 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd99406af50 msgr2=0x7fd99406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd99406af50 0x7fd99406d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd998006fd0 tx=0x7fd998009060 comp rx=0 tx=0).stop 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 msgr2=0x7fd9a8147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 0x7fd9a8147c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097d0 tx=0x7fd99c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 shutdown_connections 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd99406af50 0x7fd99406d410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8157920 0x7fd9a80cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a8156430 0x7fd9a8147c50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 --2- 172.21.15.87:0/1023677375 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a81506f0 0x7fd9a80c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 >> 172.21.15.87:0/1023677375 conn(0x7fd9a8072810 msgr2=0x7fd9a8155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:15.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 shutdown_connections 2024-01-20T08:31:15.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.012+0000 7fd9adbea640 1 -- 172.21.15.87:0/1023677375 wait complete. 2024-01-20T08:31:15.029 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-01-20T08:31:15.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.130+0000 7f1a306fb640 1 Processor -- start 2024-01-20T08:31:15.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.130+0000 7f1a306fb640 1 -- start start 2024-01-20T08:31:15.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.131+0000 7f1a306fb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a28153a90 0x7f1a2815a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:15.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.131+0000 7f1a306fb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280c3130 con 0x7f1a2815d520 2024-01-20T08:31:15.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.131+0000 7f1a306fb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280c3410 con 0x7f1a280c7b40 2024-01-20T08:31:15.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.131+0000 7f1a306fb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280c36f0 con 0x7f1a28153a90 2024-01-20T08:31:15.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.132+0000 7f1a2ec98640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1a2815d520 0x7f1a280c9200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52798/0 (socket says 172.21.15.87:52798) 2024-01-20T08:31:15.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.132+0000 7f1a2ec98640 1 -- 172.21.15.87:0/2759319740 learned_addr learned my addr 172.21.15.87:0/2759319740 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:15.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.132+0000 7f1a2e497640 1 --2- 172.21.15.87:0/2759319740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a28153a90 0x7f1a2815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:15.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876478084 0 0) 0x7f1a280c3410 con 0x7f1a280c7b40 2024-01-20T08:31:15.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a14003520 con 0x7f1a280c7b40 2024-01-20T08:31:15.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631829383 0 0) 0x7f1a280c3130 con 0x7f1a2815d520 2024-01-20T08:31:15.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a280c3410 con 0x7f1a2815d520 2024-01-20T08:31:15.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2e497640 1 -- 172.21.15.87:0/2759319740 >> v1:172.21.15.116:6789/0 conn(0x7f1a280c7b40 legacy=0x7f1a2815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2e497640 1 -- 172.21.15.87:0/2759319740 >> v1:172.21.15.87:6789/0 conn(0x7f1a2815d520 legacy=0x7f1a280c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.134+0000 7f1a2e497640 1 -- 172.21.15.87:0/2759319740 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280c39d0 con 0x7f1a28153a90 2024-01-20T08:31:15.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.136+0000 7f1a2e497640 1 --2- 172.21.15.87:0/2759319740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a28153a90 0x7f1a2815a080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009dc0 tx=0x7f1a2402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1844bc89b0597870 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:15.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.140+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2402cd50 con 0x7f1a28153a90 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.140+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24034440 con 0x7f1a28153a90 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.140+0000 7f1a2d495640 1 -- 172.21.15.87:0/2759319740 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240347d0 con 0x7f1a28153a90 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.140+0000 7f1a306fb640 1 -- 172.21.15.87:0/2759319740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a28153a90 msgr2=0x7f1a2815a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.140+0000 7f1a306fb640 1 --2- 172.21.15.87:0/2759319740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a28153a90 0x7f1a2815a080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009dc0 tx=0x7f1a2402bfa0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.141+0000 7f1a306fb640 1 -- 172.21.15.87:0/2759319740 shutdown_connections 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.141+0000 7f1a306fb640 1 --2- 172.21.15.87:0/2759319740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a28153a90 0x7f1a2815a080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.141+0000 7f1a306fb640 1 -- 172.21.15.87:0/2759319740 >> 172.21.15.87:0/2759319740 conn(0x7f1a2806a5b0 msgr2=0x7f1a2806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.141+0000 7f1a306fb640 1 -- 172.21.15.87:0/2759319740 shutdown_connections 2024-01-20T08:31:15.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.141+0000 7f1a306fb640 1 -- 172.21.15.87:0/2759319740 wait complete. 2024-01-20T08:31:15.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.142+0000 7f1a306fb640 1 Processor -- start 2024-01-20T08:31:15.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.142+0000 7f1a306fb640 1 -- start start 2024-01-20T08:31:15.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.142+0000 7f1a306fb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 0x7f1a281e5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:15.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a306fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a28153a90 0x7f1a2815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:15.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a306fb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a2815d520 0x7f1a281e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:15.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a306fb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281548f0 con 0x7f1a280c7b40 2024-01-20T08:31:15.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a306fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281550d0 con 0x7f1a28153a90 2024-01-20T08:31:15.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a306fb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281e39c0 con 0x7f1a2815d520 2024-01-20T08:31:15.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a2e497640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 0x7f1a281e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:15.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a2ec98640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a2815d520 0x7f1a281e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:15.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a2dc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a28153a90 0x7f1a2815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:15.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a2e497640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 0x7f1a281e5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56426/0 (socket says 172.21.15.87:56426) 2024-01-20T08:31:15.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.143+0000 7f1a2e497640 1 -- 172.21.15.87:0/4112429797 learned_addr learned my addr 172.21.15.87:0/4112429797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:15.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.146+0000 7f1a2e497640 1 -- 172.21.15.87:0/4112429797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a2815d520 msgr2=0x7f1a281e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.146+0000 7f1a2e497640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a2815d520 0x7f1a281e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.146+0000 7f1a2e497640 1 -- 172.21.15.87:0/4112429797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a28153a90 msgr2=0x7f1a2815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:15.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.146+0000 7f1a2e497640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a28153a90 0x7f1a2815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:15.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.146+0000 7f1a2e497640 1 -- 172.21.15.87:0/4112429797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009a70 con 0x7f1a280c7b40 2024-01-20T08:31:15.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.149+0000 7f1a2e497640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 0x7f1a281e5270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a24000f80 tx=0x7f1a24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:15.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.151+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24040650 con 0x7f1a280c7b40 2024-01-20T08:31:15.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.151+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a281552c0 con 0x7f1a280c7b40 2024-01-20T08:31:15.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.151+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a28154ad0 con 0x7f1a280c7b40 2024-01-20T08:31:15.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.152+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a28074290 con 0x7f1a280c7b40 2024-01-20T08:31:15.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.152+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24009450 con 0x7f1a280c7b40 2024-01-20T08:31:15.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.152+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240409b0 con 0x7f1a280c7b40 2024-01-20T08:31:15.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.396+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1a24040b10 con 0x7f1a280c7b40 2024-01-20T08:31:15.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.396+0000 7f1a1effd640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1a0c06aed0 0x7f1a0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:15.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.396+0000 7f1a2dc96640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1a0c06aed0 0x7f1a0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:15.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.397+0000 7f1a2dc96640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1a0c06aed0 0x7f1a0c06d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a2806b770 tx=0x7f1a18009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.410+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f1a2407b350 con 0x7f1a280c7b40 2024-01-20T08:31:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.411+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1a24077690 con 0x7f1a280c7b40 2024-01-20T08:31:15.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:15.585+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f1a280ca5d0 con 0x7f1a280c7b40 2024-01-20T08:31:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.122+0000 7f1a1effd640 1 -- 172.21.15.87:0/4112429797 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1a24049e60 con 0x7f1a280c7b40 2024-01-20T08:31:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:set pool 4 allow_ec_overwrites to true 2024-01-20T08:31:16.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1a0c06aed0 msgr2=0x7f1a0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:16.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1a0c06aed0 0x7f1a0c06d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a2806b770 tx=0x7f1a18009040 comp rx=0 tx=0).stop 2024-01-20T08:31:16.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 msgr2=0x7f1a281e5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:16.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 0x7f1a281e5270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a24000f80 tx=0x7f1a24004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:16.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 shutdown_connections 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1a0c06aed0 0x7f1a0c06d390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a2815d520 0x7f1a281e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a28153a90 0x7f1a2815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 --2- 172.21.15.87:0/4112429797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a280c7b40 0x7f1a281e5270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 >> 172.21.15.87:0/4112429797 conn(0x7f1a2806a5b0 msgr2=0x7f1a280c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 shutdown_connections 2024-01-20T08:31:16.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.124+0000 7f1a306fb640 1 -- 172.21.15.87:0/4112429797 wait complete. 2024-01-20T08:31:16.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-01-20T08:31:16.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.272+0000 7f090cda9640 1 Processor -- start 2024-01-20T08:31:16.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.272+0000 7f090cda9640 1 -- start start 2024-01-20T08:31:16.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.273+0000 7f090cda9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0908153660 0x7f090815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:16.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.273+0000 7f090cda9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c97c0 0x7f09080c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:16.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.273+0000 7f090cda9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0908153a40 con 0x7f09080c8ec0 2024-01-20T08:31:16.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.273+0000 7f090cda9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09080c32c0 con 0x7f0908153660 2024-01-20T08:31:16.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.273+0000 7f090cda9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090806d420 con 0x7f09080c97c0 2024-01-20T08:31:16.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.276+0000 7f0905d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c97c0 0x7f09080c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:16.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.280+0000 7f090659c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0908153660 0x7f090815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:16.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.281+0000 7f0906d9d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f09080c8ec0 0x7f09080ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52146/0 (socket says 172.21.15.87:52146) 2024-01-20T08:31:16.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.281+0000 7f0906d9d640 1 -- 172.21.15.87:0/257515310 learned_addr learned my addr 172.21.15.87:0/257515310 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:16.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.283+0000 7f090659c640 1 -- 172.21.15.87:0/257515310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c97c0 msgr2=0x7f09080c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:16.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.283+0000 7f090659c640 1 --2- 172.21.15.87:0/257515310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c97c0 0x7f09080c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.283+0000 7f090659c640 1 -- 172.21.15.87:0/257515310 >> v1:172.21.15.87:6789/0 conn(0x7f09080c8ec0 legacy=0x7f09080ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:16.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.283+0000 7f090659c640 1 -- 172.21.15.87:0/257515310 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09080c3a60 con 0x7f0908153660 2024-01-20T08:31:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.287+0000 7f090659c640 1 --2- 172.21.15.87:0/257515310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908153660 0x7f090815a1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08f80097a0 tx=0x7f08f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409b0415ffe7b93a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:16.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.313+0000 7f090559a640 1 -- 172.21.15.87:0/257515310 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8031830 con 0x7f0908153660 2024-01-20T08:31:16.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.313+0000 7f090559a640 1 -- 172.21.15.87:0/257515310 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8031990 con 0x7f0908153660 2024-01-20T08:31:16.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.313+0000 7f090559a640 1 -- 172.21.15.87:0/257515310 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8031d20 con 0x7f0908153660 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 -- 172.21.15.87:0/257515310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908153660 msgr2=0x7f090815a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 --2- 172.21.15.87:0/257515310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908153660 0x7f090815a1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08f80097a0 tx=0x7f08f802b8d0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 -- 172.21.15.87:0/257515310 shutdown_connections 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 --2- 172.21.15.87:0/257515310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c97c0 0x7f09080c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 --2- 172.21.15.87:0/257515310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908153660 0x7f090815a1b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 -- 172.21.15.87:0/257515310 >> 172.21.15.87:0/257515310 conn(0x7f090806a5b0 msgr2=0x7f090806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 -- 172.21.15.87:0/257515310 shutdown_connections 2024-01-20T08:31:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.314+0000 7f090cda9640 1 -- 172.21.15.87:0/257515310 wait complete. 2024-01-20T08:31:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.315+0000 7f090cda9640 1 Processor -- start 2024-01-20T08:31:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.315+0000 7f090cda9640 1 -- start start 2024-01-20T08:31:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.316+0000 7f090cda9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 0x7f0908148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.316+0000 7f090cda9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09080c97c0 0x7f0908148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.316+0000 7f090cda9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908153660 0x7f090814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.316+0000 7f090cda9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090814ca20 con 0x7f09080c97c0 2024-01-20T08:31:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.316+0000 7f090cda9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0908146860 con 0x7f0908153660 2024-01-20T08:31:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.316+0000 7f090cda9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090814db80 con 0x7f09080c8ec0 2024-01-20T08:31:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.317+0000 7f0905d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09080c97c0 0x7f0908148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.317+0000 7f0906d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908153660 0x7f090814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.317+0000 7f090659c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 0x7f0908148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.317+0000 7f090659c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 0x7f0908148770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:55892/0 (socket says 172.21.15.87:55892) 2024-01-20T08:31:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.317+0000 7f090659c640 1 -- 172.21.15.87:0/2682361547 learned_addr learned my addr 172.21.15.87:0/2682361547 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:16.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.320+0000 7f090659c640 1 -- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908153660 msgr2=0x7f090814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:16.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.321+0000 7f090659c640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908153660 0x7f090814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.321+0000 7f090659c640 1 -- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09080c97c0 msgr2=0x7f0908148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:16.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.321+0000 7f090659c640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09080c97c0 0x7f0908148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:16.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.321+0000 7f090659c640 1 -- 172.21.15.87:0/2682361547 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8009400 con 0x7f09080c8ec0 2024-01-20T08:31:16.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.323+0000 7f090659c640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 0x7f0908148770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08f802bee0 tx=0x7f08f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:16.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.327+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f803ba70 con 0x7f09080c8ec0 2024-01-20T08:31:16.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.327+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f803bbf0 con 0x7f09080c8ec0 2024-01-20T08:31:16.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.327+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090814e140 con 0x7f09080c8ec0 2024-01-20T08:31:16.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.327+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80453f0 con 0x7f09080c8ec0 2024-01-20T08:31:16.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.327+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0908146a90 con 0x7f09080c8ec0 2024-01-20T08:31:16.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.328+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0908074290 con 0x7f09080c8ec0 2024-01-20T08:31:16.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.345+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f08f804e450 con 0x7f09080c8ec0 2024-01-20T08:31:16.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.346+0000 7f08f6ffd640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08e806af20 0x7f08e806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:16.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.346+0000 7f0905d9b640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08e806af20 0x7f08e806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:16.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.346+0000 7f0905d9b640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08e806af20 0x7f08e806d3e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f09080cd730 tx=0x7f08fc008040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:16.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.348+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f08f807a820 con 0x7f09080c8ec0 2024-01-20T08:31:16.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.585+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f08f8076b60 con 0x7f09080c8ec0 2024-01-20T08:31:16.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:16.792+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f090815da50 con 0x7f09080c8ec0 2024-01-20T08:31:17.029 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-01-20T08:31:17.022+0000 f5d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-20T08:31:17.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.277+0000 7f08f6ffd640 1 -- 172.21.15.87:0/2682361547 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f08f807a180 con 0x7f09080c8ec0 2024-01-20T08:31:17.281 INFO:teuthology.orchestra.run.smithi087.stderr:added data pool 4 to fsmap 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08e806af20 msgr2=0x7f08e806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08e806af20 0x7f08e806d3e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f09080cd730 tx=0x7f08fc008040 comp rx=0 tx=0).stop 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 msgr2=0x7f0908148770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 0x7f0908148770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08f802bee0 tx=0x7f08f80046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 shutdown_connections 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08e806af20 0x7f08e806d3e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908153660 0x7f090814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09080c97c0 0x7f0908148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 --2- 172.21.15.87:0/2682361547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09080c8ec0 0x7f0908148770 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.279+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 >> 172.21.15.87:0/2682361547 conn(0x7f090806a5b0 msgr2=0x7f0908157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:17.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.280+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 shutdown_connections 2024-01-20T08:31:17.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.280+0000 7f090cda9640 1 -- 172.21.15.87:0/2682361547 wait complete. 2024-01-20T08:31:17.293 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:17.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.391+0000 7f63bc2a2640 1 Processor -- start 2024-01-20T08:31:17.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.391+0000 7f63bc2a2640 1 -- start start 2024-01-20T08:31:17.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.392+0000 7f63bc2a2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b406b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:17.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.392+0000 7f63bc2a2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63b406fd60 0x7f63b406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:17.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.392+0000 7f63bc2a2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b4151b10 con 0x7f63b4151730 2024-01-20T08:31:17.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.392+0000 7f63bc2a2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b41581a0 con 0x7f63b406fd60 2024-01-20T08:31:17.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.392+0000 7f63bc2a2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b406bec0 con 0x7f63b4072580 2024-01-20T08:31:17.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.393+0000 7f63ba83f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63b406fd60 0x7f63b406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:17.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.393+0000 7f63ba03e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f63b4151730 0x7f63b40c49e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38454/0 (socket says 172.21.15.87:38454) 2024-01-20T08:31:17.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.393+0000 7f63ba03e640 1 -- 172.21.15.87:0/3445885358 learned_addr learned my addr 172.21.15.87:0/3445885358 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:17.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.393+0000 7f63b983d640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:17.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63b903c640 1 -- 172.21.15.87:0/3445885358 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570021991 0 0) 0x7f63b0001490 con 0x7f63b4151730 2024-01-20T08:31:17.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63b903c640 1 -- 172.21.15.87:0/3445885358 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a0003520 con 0x7f63b4151730 2024-01-20T08:31:17.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63ba83f640 1 -- 172.21.15.87:0/3445885358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 msgr2=0x7f63b406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:17.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63ba83f640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b406b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63ba83f640 1 -- 172.21.15.87:0/3445885358 >> v1:172.21.15.116:6789/0 conn(0x7f63b4151730 legacy=0x7f63b40c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63ba83f640 1 -- 172.21.15.87:0/3445885358 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4072a50 con 0x7f63b406fd60 2024-01-20T08:31:17.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.395+0000 7f63b983d640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:17.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.397+0000 7f63ba83f640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b406fd60 0x7f63b406a640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097d0 tx=0x7f63ac02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cc5de0631f1475 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.401+0000 7f63b903c640 1 -- 172.21.15.87:0/3445885358 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac031890 con 0x7f63b406fd60 2024-01-20T08:31:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.401+0000 7f63b903c640 1 -- 172.21.15.87:0/3445885358 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac0319f0 con 0x7f63b406fd60 2024-01-20T08:31:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.401+0000 7f63b903c640 1 -- 172.21.15.87:0/3445885358 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac031d80 con 0x7f63b406fd60 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.401+0000 7f63bc2a2640 1 -- 172.21.15.87:0/3445885358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b406fd60 msgr2=0x7f63b406a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.401+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b406fd60 0x7f63b406a640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097d0 tx=0x7f63ac02b930 comp rx=0 tx=0).stop 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.402+0000 7f63bc2a2640 1 -- 172.21.15.87:0/3445885358 shutdown_connections 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.402+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b406fd60 0x7f63b406a640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.402+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/3445885358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.402+0000 7f63bc2a2640 1 -- 172.21.15.87:0/3445885358 >> 172.21.15.87:0/3445885358 conn(0x7f63b4071690 msgr2=0x7f63b4071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.402+0000 7f63bc2a2640 1 -- 172.21.15.87:0/3445885358 shutdown_connections 2024-01-20T08:31:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.402+0000 7f63bc2a2640 1 -- 172.21.15.87:0/3445885358 wait complete. 2024-01-20T08:31:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 Processor -- start 2024-01-20T08:31:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 -- start start 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 0x7f63b414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b41f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b4151730 0x7f63b41f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4159530 con 0x7f63b4151730 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4151cb0 con 0x7f63b406fd60 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.403+0000 7f63bc2a2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4151ee0 con 0x7f63b4072580 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.404+0000 7f63ba83f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b4151730 0x7f63b41f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.404+0000 7f63ba03e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 0x7f63b414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.404+0000 7f63b983d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b41f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.404+0000 7f63ba03e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 0x7f63b414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41082/0 (socket says 172.21.15.87:41082) 2024-01-20T08:31:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.404+0000 7f63ba03e640 1 -- 172.21.15.87:0/4009521364 learned_addr learned my addr 172.21.15.87:0/4009521364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:17.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.407+0000 7f63ba03e640 1 -- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 msgr2=0x7f63b41f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.407+0000 7f63ba03e640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b41f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.407+0000 7f63ba03e640 1 -- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b4151730 msgr2=0x7f63b41f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.407+0000 7f63ba03e640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b4151730 0x7f63b41f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.407+0000 7f63ba03e640 1 -- 172.21.15.87:0/4009521364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4009400 con 0x7f63b406fd60 2024-01-20T08:31:17.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.410+0000 7f63ba03e640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 0x7f63b414f900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b000cb10 tx=0x7f63b000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:17.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.413+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b000ac70 con 0x7f63b406fd60 2024-01-20T08:31:17.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.413+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b000adf0 con 0x7f63b406fd60 2024-01-20T08:31:17.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.413+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac009400 con 0x7f63b406fd60 2024-01-20T08:31:17.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.413+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0012640 con 0x7f63b406fd60 2024-01-20T08:31:17.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.413+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b41524a0 con 0x7f63b406fd60 2024-01-20T08:31:17.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.414+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4074290 con 0x7f63b406fd60 2024-01-20T08:31:17.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.434+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f63b0007070 con 0x7f63b406fd60 2024-01-20T08:31:17.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.435+0000 7f63aaffd640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639806af20 0x7f639806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:17.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.435+0000 7f63b983d640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639806af20 0x7f639806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:17.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.435+0000 7f63b983d640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639806af20 0x7f639806d3e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63b4150320 tx=0x7f63a4009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:17.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.659+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f63b0058410 con 0x7f63b406fd60 2024-01-20T08:31:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.669+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f63b0054750 con 0x7f63b406fd60 2024-01-20T08:31:17.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.873+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63b4151b10 con 0x7f63b406fd60 2024-01-20T08:31:17.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.899+0000 7f63aaffd640 1 -- 172.21.15.87:0/4009521364 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6517 (secure 0 0 0) 0x7f63b4151b10 con 0x7f63b406fd60 2024-01-20T08:31:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:17.902 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:16.816824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-01-20T08:31:17.902 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 7 2024-01-20T08:31:17.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639806af20 msgr2=0x7f639806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639806af20 0x7f639806d3e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63b4150320 tx=0x7f63a4009190 comp rx=0 tx=0).stop 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 msgr2=0x7f63b414f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 0x7f63b414f900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b000cb10 tx=0x7f63b000a060 comp rx=0 tx=0).stop 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 shutdown_connections 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639806af20 0x7f639806d3e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63b4151730 0x7f63b41f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63b4072580 0x7f63b41f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 --2- 172.21.15.87:0/4009521364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406fd60 0x7f63b414f900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 >> 172.21.15.87:0/4009521364 conn(0x7f63b4071690 msgr2=0x7f63b40c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 shutdown_connections 2024-01-20T08:31:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:17.900+0000 7f63bc2a2640 1 -- 172.21.15.87:0/4009521364 wait complete. 2024-01-20T08:31:17.914 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:18.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.010+0000 7f456bd5a640 1 Processor -- start 2024-01-20T08:31:18.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.010+0000 7f456bd5a640 1 -- start start 2024-01-20T08:31:18.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.011+0000 7f456bd5a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45640c9920 0x7f45641539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.011+0000 7f456bd5a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456415d600 con 0x7f456415da90 2024-01-20T08:31:18.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.011+0000 7f456bd5a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45640cfe00 con 0x7f45641535c0 2024-01-20T08:31:18.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.011+0000 7f456bd5a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456415a820 con 0x7f45640c9920 2024-01-20T08:31:18.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.012+0000 7f456a2f7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45640c9920 0x7f45641539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.012+0000 7f4569af6640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f45641535c0 0x7f45640ca200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38462/0 (socket says 172.21.15.87:38462) 2024-01-20T08:31:18.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.012+0000 7f4569af6640 1 -- 172.21.15.87:0/3429520051 learned_addr learned my addr 172.21.15.87:0/3429520051 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:18.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.015+0000 7f456a2f7640 1 -- 172.21.15.87:0/3429520051 >> v1:172.21.15.116:6789/0 conn(0x7f45641535c0 legacy=0x7f45640ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.015+0000 7f456a2f7640 1 -- 172.21.15.87:0/3429520051 >> v1:172.21.15.87:6789/0 conn(0x7f456415da90 legacy=0x7f4564162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:18.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.015+0000 7f456a2f7640 1 -- 172.21.15.87:0/3429520051 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4564159fa0 con 0x7f45640c9920 2024-01-20T08:31:18.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.020+0000 7f456a2f7640 1 --2- 172.21.15.87:0/3429520051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45640c9920 0x7f45641539a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f455c0097a0 tx=0x7f455c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61f9d23f290341f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:18.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.023+0000 7f4568af4640 1 -- 172.21.15.87:0/3429520051 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c030610 con 0x7f45640c9920 2024-01-20T08:31:18.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.023+0000 7f4568af4640 1 -- 172.21.15.87:0/3429520051 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c030770 con 0x7f45640c9920 2024-01-20T08:31:18.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.023+0000 7f4568af4640 1 -- 172.21.15.87:0/3429520051 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c030b00 con 0x7f45640c9920 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 -- 172.21.15.87:0/3429520051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45640c9920 msgr2=0x7f45641539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 --2- 172.21.15.87:0/3429520051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45640c9920 0x7f45641539a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f455c0097a0 tx=0x7f455c02b960 comp rx=0 tx=0).stop 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 -- 172.21.15.87:0/3429520051 shutdown_connections 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 --2- 172.21.15.87:0/3429520051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45640c9920 0x7f45641539a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 -- 172.21.15.87:0/3429520051 >> 172.21.15.87:0/3429520051 conn(0x7f456408af90 msgr2=0x7f456408b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 -- 172.21.15.87:0/3429520051 shutdown_connections 2024-01-20T08:31:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.024+0000 7f456bd5a640 1 -- 172.21.15.87:0/3429520051 wait complete. 2024-01-20T08:31:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.025+0000 7f456bd5a640 1 Processor -- start 2024-01-20T08:31:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.025+0000 7f456bd5a640 1 -- start start 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.026+0000 7f456bd5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45640c9920 0x7f456415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.026+0000 7f456bd5a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 0x7f4564158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.026+0000 7f456bd5a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f456415da90 0x7f45641574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.026+0000 7f456bd5a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641e3450 con 0x7f45640c9920 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.026+0000 7f456bd5a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641e2c40 con 0x7f45641535c0 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.026+0000 7f456bd5a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45641e46f0 con 0x7f456415da90 2024-01-20T08:31:18.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.027+0000 7f456a2f7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f456415da90 0x7f45641574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.027+0000 7f45692f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 0x7f4564158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.028+0000 7f45692f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 0x7f4564158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41088/0 (socket says 172.21.15.87:41088) 2024-01-20T08:31:18.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.028+0000 7f45692f5640 1 -- 172.21.15.87:0/44016183 learned_addr learned my addr 172.21.15.87:0/44016183 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:18.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.030+0000 7f4569af6640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45640c9920 0x7f456415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.031+0000 7f45692f5640 1 -- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f456415da90 msgr2=0x7f45641574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.031+0000 7f45692f5640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f456415da90 0x7f45641574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.031+0000 7f45692f5640 1 -- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45640c9920 msgr2=0x7f456415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.031+0000 7f45692f5640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45640c9920 0x7f456415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.031+0000 7f45692f5640 1 -- 172.21.15.87:0/44016183 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c009400 con 0x7f45641535c0 2024-01-20T08:31:18.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.034+0000 7f45692f5640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 0x7f4564158fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f455400c440 tx=0x7f455400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:18.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.038+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554005480 con 0x7f45641535c0 2024-01-20T08:31:18.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.038+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45540035f0 con 0x7f45641535c0 2024-01-20T08:31:18.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.038+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540039c0 con 0x7f45641535c0 2024-01-20T08:31:18.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.038+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4564159b80 con 0x7f45641535c0 2024-01-20T08:31:18.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.038+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45641e3170 con 0x7f45641535c0 2024-01-20T08:31:18.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.039+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640c5340 con 0x7f45641535c0 2024-01-20T08:31:18.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.058+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f455401b3b0 con 0x7f45641535c0 2024-01-20T08:31:18.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.058+0000 7f455a7fc640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f454806af20 0x7f454806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.058+0000 7f4569af6640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f454806af20 0x7f454806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.058+0000 7f4569af6640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f454806af20 0x7f454806d3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45600012e0 tx=0x7f4560009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:18.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.061+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f45540010c0 con 0x7f45641535c0 2024-01-20T08:31:18.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.303+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f45540577f0 con 0x7f45641535c0 2024-01-20T08:31:18.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.501+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45640cf0f0 con 0x7f45641535c0 2024-01-20T08:31:18.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.610+0000 7f455a7fc640 1 -- 172.21.15.87:0/44016183 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f4554017020 con 0x7f45641535c0 2024-01-20T08:31:18.613 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:18.613 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f454806af20 msgr2=0x7f454806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f454806af20 0x7f454806d3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45600012e0 tx=0x7f4560009040 comp rx=0 tx=0).stop 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 msgr2=0x7f4564158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 0x7f4564158fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f455400c440 tx=0x7f455400c900 comp rx=0 tx=0).stop 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 shutdown_connections 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f454806af20 0x7f454806d3e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f456415da90 0x7f45641574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45641535c0 0x7f4564158fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 --2- 172.21.15.87:0/44016183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45640c9920 0x7f456415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 >> 172.21.15.87:0/44016183 conn(0x7f456408af90 msgr2=0x7f45640c4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 shutdown_connections 2024-01-20T08:31:18.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.611+0000 7f456bd5a640 1 -- 172.21.15.87:0/44016183 wait complete. 2024-01-20T08:31:18.626 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:18.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.722+0000 7fd7ca35a640 1 Processor -- start 2024-01-20T08:31:18.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.722+0000 7fd7ca35a640 1 -- start start 2024-01-20T08:31:18.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.723+0000 7fd7ca35a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7c4159760 0x7fd7c40cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.723+0000 7fd7ca35a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c415d0e0 con 0x7fd7c4156e00 2024-01-20T08:31:18.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.723+0000 7fd7ca35a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c41571e0 con 0x7fd7c406f3e0 2024-01-20T08:31:18.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.723+0000 7fd7ca35a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c40c6ed0 con 0x7fd7c4159760 2024-01-20T08:31:18.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.724+0000 7fd7c37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7c4159760 0x7fd7c40cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.724+0000 7fd7c88f7640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd7c406f3e0 0x7fd7c4164000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38470/0 (socket says 172.21.15.87:38470) 2024-01-20T08:31:18.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.724+0000 7fd7c88f7640 1 -- 172.21.15.87:0/1673616642 learned_addr learned my addr 172.21.15.87:0/1673616642 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:18.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.727+0000 7fd7c37fe640 1 -- 172.21.15.87:0/1673616642 >> v1:172.21.15.116:6789/0 conn(0x7fd7c406f3e0 legacy=0x7fd7c4164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.727+0000 7fd7c37fe640 1 -- 172.21.15.87:0/1673616642 >> v1:172.21.15.87:6789/0 conn(0x7fd7c4156e00 legacy=0x7fd7c415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.727+0000 7fd7c37fe640 1 -- 172.21.15.87:0/1673616642 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40720c0 con 0x7fd7c4159760 2024-01-20T08:31:18.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.731+0000 7fd7c37fe640 1 --2- 172.21.15.87:0/1673616642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c4159760 0x7fd7c40cb000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80097a0 tx=0x7fd7b802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414cf483415f0072 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:18.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.735+0000 7fd7c2ffd640 1 -- 172.21.15.87:0/1673616642 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b802f810 con 0x7fd7c4159760 2024-01-20T08:31:18.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.735+0000 7fd7c2ffd640 1 -- 172.21.15.87:0/1673616642 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b802f970 con 0x7fd7c4159760 2024-01-20T08:31:18.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.735+0000 7fd7c2ffd640 1 -- 172.21.15.87:0/1673616642 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b802fd00 con 0x7fd7c4159760 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 -- 172.21.15.87:0/1673616642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c4159760 msgr2=0x7fd7c40cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/1673616642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c4159760 0x7fd7c40cb000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80097a0 tx=0x7fd7b802b9f0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 -- 172.21.15.87:0/1673616642 shutdown_connections 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/1673616642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c4159760 0x7fd7c40cb000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 -- 172.21.15.87:0/1673616642 >> 172.21.15.87:0/1673616642 conn(0x7fd7c406d510 msgr2=0x7fd7c40663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 -- 172.21.15.87:0/1673616642 shutdown_connections 2024-01-20T08:31:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.736+0000 7fd7ca35a640 1 -- 172.21.15.87:0/1673616642 wait complete. 2024-01-20T08:31:18.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.737+0000 7fd7ca35a640 1 Processor -- start 2024-01-20T08:31:18.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.737+0000 7fd7ca35a640 1 -- start start 2024-01-20T08:31:18.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7ca35a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 0x7fd7c4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7ca35a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7c4156e00 0x7fd7c4155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7ca35a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7c4159760 0x7fd7c41f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7ca35a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4159620 con 0x7fd7c406f3e0 2024-01-20T08:31:18.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7ca35a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415c000 con 0x7fd7c4159760 2024-01-20T08:31:18.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7ca35a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415aa60 con 0x7fd7c4156e00 2024-01-20T08:31:18.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.738+0000 7fd7c3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 0x7fd7c4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.739+0000 7fd7c37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7c4156e00 0x7fd7c4155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.739+0000 7fd7c3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 0x7fd7c4159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38208/0 (socket says 172.21.15.87:38208) 2024-01-20T08:31:18.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.739+0000 7fd7c3fff640 1 -- 172.21.15.87:0/973510944 learned_addr learned my addr 172.21.15.87:0/973510944 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:18.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.740+0000 7fd7c88f7640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7c4159760 0x7fd7c41f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.742+0000 7fd7c3fff640 1 -- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7c4156e00 msgr2=0x7fd7c4155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.742+0000 7fd7c3fff640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7c4156e00 0x7fd7c4155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.742+0000 7fd7c3fff640 1 -- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7c4159760 msgr2=0x7fd7c41f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:18.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.742+0000 7fd7c3fff640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7c4159760 0x7fd7c41f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:18.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.742+0000 7fd7c3fff640 1 -- 172.21.15.87:0/973510944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8009400 con 0x7fd7c406f3e0 2024-01-20T08:31:18.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.745+0000 7fd7c3fff640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 0x7fd7c4159000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c430 tx=0x7fd7b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:18.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.749+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40052d0 con 0x7fd7c406f3e0 2024-01-20T08:31:18.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.749+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4005450 con 0x7fd7c406f3e0 2024-01-20T08:31:18.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.749+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4007260 con 0x7fd7c406f3e0 2024-01-20T08:31:18.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.749+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c415ad40 con 0x7fd7c406f3e0 2024-01-20T08:31:18.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.749+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c415b300 con 0x7fd7c406f3e0 2024-01-20T08:31:18.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.751+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c41f2b40 con 0x7fd7c406f3e0 2024-01-20T08:31:18.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.769+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd7b40073c0 con 0x7fd7c406f3e0 2024-01-20T08:31:18.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.770+0000 7fd7c0ff9640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af20 0x7fd7b006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:18.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.770+0000 7fd7c37fe640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af20 0x7fd7b006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:18.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.770+0000 7fd7c37fe640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af20 0x7fd7b006d3e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8036000 tx=0x7fd7b8008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:18.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:18.773+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd7b4057950 con 0x7fd7c406f3e0 2024-01-20T08:31:19.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.006+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd7b4053c90 con 0x7fd7c406f3e0 2024-01-20T08:31:19.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.180+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7c41f2b40 con 0x7fd7c406f3e0 2024-01-20T08:31:19.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.255+0000 7fd7c0ff9640 1 -- 172.21.15.87:0/973510944 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fd7c41f2b40 con 0x7fd7c406f3e0 2024-01-20T08:31:19.259 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:19.259 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:19.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af20 msgr2=0x7fd7b006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af20 0x7fd7b006d3e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8036000 tx=0x7fd7b8008030 comp rx=0 tx=0).stop 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 msgr2=0x7fd7c4159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 0x7fd7c4159000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c430 tx=0x7fd7b400c8f0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 shutdown_connections 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af20 0x7fd7b006d3e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7c4159760 0x7fd7c41f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7c4156e00 0x7fd7c4155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 --2- 172.21.15.87:0/973510944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c406f3e0 0x7fd7c4159000 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 >> 172.21.15.87:0/973510944 conn(0x7fd7c406d510 msgr2=0x7fd7c4159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 shutdown_connections 2024-01-20T08:31:19.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.258+0000 7fd7ca35a640 1 -- 172.21.15.87:0/973510944 wait complete. 2024-01-20T08:31:19.286 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:19.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.387+0000 7f9b8a390640 1 Processor -- start 2024-01-20T08:31:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.387+0000 7f9b8a390640 1 -- start start 2024-01-20T08:31:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.388+0000 7f9b8a390640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 0x7f9b84157560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.388+0000 7f9b8a390640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9b840c7e30 0x7f9b840c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.388+0000 7f9b8a390640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b84157b80 con 0x7f9b84151800 2024-01-20T08:31:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.388+0000 7f9b8a390640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b84157e60 con 0x7f9b840c7e30 2024-01-20T08:31:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.388+0000 7f9b8a390640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84158140 con 0x7f9b840ca310 2024-01-20T08:31:19.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.402+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 0x7f9b84157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:19.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.404+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 0x7f9b84157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:55964/0 (socket says 172.21.15.87:55964) 2024-01-20T08:31:19.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.404+0000 7f9b837fe640 1 -- 172.21.15.87:0/803885578 learned_addr learned my addr 172.21.15.87:0/803885578 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:19.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.411+0000 7f9b8892d640 1 --2- 172.21.15.87:0/803885578 >> v2:172.21.15.116:3300/0 conn(0x7f9b840c7e30 0x7f9b840c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:19.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.411+0000 7f9b837fe640 1 -- 172.21.15.87:0/803885578 >> v2:172.21.15.116:3300/0 conn(0x7f9b840c7e30 msgr2=0x7f9b840c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:19.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.411+0000 7f9b837fe640 1 --2- 172.21.15.87:0/803885578 >> v2:172.21.15.116:3300/0 conn(0x7f9b840c7e30 0x7f9b840c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.411+0000 7f9b837fe640 1 -- 172.21.15.87:0/803885578 >> v1:172.21.15.116:6789/0 conn(0x7f9b84151800 legacy=0x7f9b8415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:19.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.411+0000 7f9b837fe640 1 -- 172.21.15.87:0/803885578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b84158420 con 0x7f9b840ca310 2024-01-20T08:31:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.414+0000 7f9b837fe640 1 --2- 172.21.15.87:0/803885578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 0x7f9b84157560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098a0 tx=0x7f9b7802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c6ebdd5838d1d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:19.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b82ffd640 1 -- 172.21.15.87:0/803885578 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78030840 con 0x7f9b840ca310 2024-01-20T08:31:19.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b82ffd640 1 -- 172.21.15.87:0/803885578 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b780309a0 con 0x7f9b840ca310 2024-01-20T08:31:19.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b82ffd640 1 -- 172.21.15.87:0/803885578 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78030d30 con 0x7f9b840ca310 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b8a390640 1 -- 172.21.15.87:0/803885578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 msgr2=0x7f9b84157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b8a390640 1 --2- 172.21.15.87:0/803885578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 0x7f9b84157560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098a0 tx=0x7f9b7802b980 comp rx=0 tx=0).stop 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b8a390640 1 -- 172.21.15.87:0/803885578 shutdown_connections 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b8a390640 1 --2- 172.21.15.87:0/803885578 >> v2:172.21.15.116:3300/0 conn(0x7f9b840c7e30 0x7f9b840c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.418+0000 7f9b8a390640 1 --2- 172.21.15.87:0/803885578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b840ca310 0x7f9b84157560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.419+0000 7f9b8a390640 1 -- 172.21.15.87:0/803885578 >> 172.21.15.87:0/803885578 conn(0x7f9b8406eea0 msgr2=0x7f9b8406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.419+0000 7f9b8a390640 1 -- 172.21.15.87:0/803885578 shutdown_connections 2024-01-20T08:31:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.419+0000 7f9b8a390640 1 -- 172.21.15.87:0/803885578 wait complete. 2024-01-20T08:31:19.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.420+0000 7f9b8a390640 1 Processor -- start 2024-01-20T08:31:19.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.420+0000 7f9b8a390640 1 -- start start 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b8a390640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b840c7e30 0x7f9b840cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b8a390640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 0x7f9b8414ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b8a390640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b84151800 0x7f9b840c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b8a390640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84157590 con 0x7f9b840c7e30 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b8a390640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841ef220 con 0x7f9b840ca310 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b8a390640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cbcb0 con 0x7f9b84151800 2024-01-20T08:31:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 0x7f9b8414ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:19.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 0x7f9b8414ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41126/0 (socket says 172.21.15.87:41126) 2024-01-20T08:31:19.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.421+0000 7f9b837fe640 1 -- 172.21.15.87:0/2898590390 learned_addr learned my addr 172.21.15.87:0/2898590390 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:19.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.422+0000 7f9b8892d640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b84151800 0x7f9b840c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:19.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.426+0000 7f9b837fe640 1 -- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b84151800 msgr2=0x7f9b840c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.426+0000 7f9b837fe640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b84151800 0x7f9b840c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.426+0000 7f9b837fe640 1 -- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b840c7e30 msgr2=0x7f9b840cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.426+0000 7f9b837fe640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b840c7e30 0x7f9b840cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.426+0000 7f9b837fe640 1 -- 172.21.15.87:0/2898590390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b780094d0 con 0x7f9b840ca310 2024-01-20T08:31:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.429+0000 7f9b837fe640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 0x7f9b8414ff20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9b780099c0 tx=0x7f9b78004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.433+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78044480 con 0x7f9b840ca310 2024-01-20T08:31:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.433+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78044600 con 0x7f9b840ca310 2024-01-20T08:31:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.433+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780449b0 con 0x7f9b840ca310 2024-01-20T08:31:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.433+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b841f3640 con 0x7f9b840ca310 2024-01-20T08:31:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.433+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b841ef430 con 0x7f9b840ca310 2024-01-20T08:31:19.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.434+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84074290 con 0x7f9b840ca310 2024-01-20T08:31:19.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.452+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9b7804d3b0 con 0x7f9b840ca310 2024-01-20T08:31:19.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.452+0000 7f9b80ff9640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9b7006af50 0x7f9b7006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:19.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.452+0000 7f9b83fff640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9b7006af50 0x7f9b7006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:19.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.452+0000 7f9b83fff640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9b7006af50 0x7f9b7006d410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b74006fd0 tx=0x7f9b74006bc0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:19.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.455+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9b7803e070 con 0x7f9b840ca310 2024-01-20T08:31:19.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.691+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9b78077910 con 0x7f9b840ca310 2024-01-20T08:31:19.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.900+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b840b21d0 con 0x7f9b840ca310 2024-01-20T08:31:19.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.912+0000 7f9b80ff9640 1 -- 172.21.15.87:0/2898590390 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6517 (secure 0 0 0) 0x7f9b78049090 con 0x7f9b840ca310 2024-01-20T08:31:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:19.915 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:16.816824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-01-20T08:31:19.916 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 7 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9b7006af50 msgr2=0x7f9b7006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9b7006af50 0x7f9b7006d410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b74006fd0 tx=0x7f9b74006bc0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 msgr2=0x7f9b8414ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 0x7f9b8414ff20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9b780099c0 tx=0x7f9b78004730 comp rx=0 tx=0).stop 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 shutdown_connections 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9b7006af50 0x7f9b7006d410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b84151800 0x7f9b840c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b840ca310 0x7f9b8414ff20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 --2- 172.21.15.87:0/2898590390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b840c7e30 0x7f9b840cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 >> 172.21.15.87:0/2898590390 conn(0x7f9b8406eea0 msgr2=0x7f9b8406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 shutdown_connections 2024-01-20T08:31:19.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:19.914+0000 7f9b8a390640 1 -- 172.21.15.87:0/2898590390 wait complete. 2024-01-20T08:31:19.928 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:20.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.024+0000 7fe1e7216640 1 Processor -- start 2024-01-20T08:31:20.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.024+0000 7fe1e7216640 1 -- start start 2024-01-20T08:31:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.026+0000 7fe1e7216640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1e0151740 0x7fe1e00c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.026+0000 7fe1e7216640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1e00ca0e0 0x7fe1e00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.026+0000 7fe1e7216640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e00c22d0 con 0x7fe1e0158100 2024-01-20T08:31:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.026+0000 7fe1e7216640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e0157450 con 0x7fe1e00ca0e0 2024-01-20T08:31:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.026+0000 7fe1e7216640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e015b930 con 0x7fe1e0151740 2024-01-20T08:31:20.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.028+0000 7fe1e4fb2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1e0151740 0x7fe1e00c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:20.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.028+0000 7fe1e4fb2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1e0151740 0x7fe1e00c1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41134/0 (socket says 172.21.15.87:41134) 2024-01-20T08:31:20.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.028+0000 7fe1e4fb2640 1 -- 172.21.15.87:0/2592604012 learned_addr learned my addr 172.21.15.87:0/2592604012 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:20.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.031+0000 7fe1e4fb2640 1 -- 172.21.15.87:0/2592604012 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00ca0e0 msgr2=0x7fe1e00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:20.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.031+0000 7fe1e4fb2640 1 --2- 172.21.15.87:0/2592604012 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00ca0e0 0x7fe1e00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.031+0000 7fe1e4fb2640 1 -- 172.21.15.87:0/2592604012 >> v1:172.21.15.87:6789/0 conn(0x7fe1e0158100 legacy=0x7fe1e015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:20.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.031+0000 7fe1e4fb2640 1 -- 172.21.15.87:0/2592604012 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e0157660 con 0x7fe1e0151740 2024-01-20T08:31:20.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.032+0000 7fe1e4fb2640 1 --2- 172.21.15.87:0/2592604012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0151740 0x7fe1e00c1d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1e006daf0 tx=0x7fe1d402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da4b91a1c56d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.036+0000 7fe1df7fe640 1 -- 172.21.15.87:0/2592604012 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4031870 con 0x7fe1e0151740 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.036+0000 7fe1df7fe640 1 -- 172.21.15.87:0/2592604012 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d40319d0 con 0x7fe1e0151740 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.036+0000 7fe1df7fe640 1 -- 172.21.15.87:0/2592604012 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4031d60 con 0x7fe1e0151740 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 -- 172.21.15.87:0/2592604012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0151740 msgr2=0x7fe1e00c1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 --2- 172.21.15.87:0/2592604012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0151740 0x7fe1e00c1d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1e006daf0 tx=0x7fe1d402b910 comp rx=0 tx=0).stop 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 -- 172.21.15.87:0/2592604012 shutdown_connections 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 --2- 172.21.15.87:0/2592604012 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00ca0e0 0x7fe1e00cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 --2- 172.21.15.87:0/2592604012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0151740 0x7fe1e00c1d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 -- 172.21.15.87:0/2592604012 >> 172.21.15.87:0/2592604012 conn(0x7fe1e006a5b0 msgr2=0x7fe1e006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:20.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 -- 172.21.15.87:0/2592604012 shutdown_connections 2024-01-20T08:31:20.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.037+0000 7fe1e7216640 1 -- 172.21.15.87:0/2592604012 wait complete. 2024-01-20T08:31:20.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.038+0000 7fe1e7216640 1 Processor -- start 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.038+0000 7fe1e7216640 1 -- start start 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e7216640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e00ca0e0 0x7fe1e006d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e7216640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe1e0151740 0x7fe1e0071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e7216640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 0x7fe1e006c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e7216640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00b2670 con 0x7fe1e00ca0e0 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e7216640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00b20d0 con 0x7fe1e0158100 2024-01-20T08:31:20.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e7216640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00b2390 con 0x7fe1e0151740 2024-01-20T08:31:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e57b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 0x7fe1e006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1dffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe1e0151740 0x7fe1e0071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e57b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 0x7fe1e006c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41148/0 (socket says 172.21.15.87:41148) 2024-01-20T08:31:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.039+0000 7fe1e57b3640 1 -- 172.21.15.87:0/548343998 learned_addr learned my addr 172.21.15.87:0/548343998 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:20.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.043+0000 7fe1e57b3640 1 -- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe1e0151740 msgr2=0x7fe1e0071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:20.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.043+0000 7fe1e57b3640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe1e0151740 0x7fe1e0071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.043+0000 7fe1e57b3640 1 -- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e00ca0e0 msgr2=0x7fe1e006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:20.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.043+0000 7fe1e57b3640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e00ca0e0 0x7fe1e006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.043+0000 7fe1e57b3640 1 -- 172.21.15.87:0/548343998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40094d0 con 0x7fe1e0158100 2024-01-20T08:31:20.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.045+0000 7fe1e57b3640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 0x7fe1e006c650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b970 tx=0x7fe1d800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:20.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.050+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800cb40 con 0x7fe1e0158100 2024-01-20T08:31:20.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.050+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d800ccc0 con 0x7fe1e0158100 2024-01-20T08:31:20.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.050+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8014680 con 0x7fe1e0158100 2024-01-20T08:31:20.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.050+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e006bb70 con 0x7fe1e0158100 2024-01-20T08:31:20.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.050+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e006c130 con 0x7fe1e0158100 2024-01-20T08:31:20.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.051+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e006c350 con 0x7fe1e0158100 2024-01-20T08:31:20.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.297+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe1d801d3b0 con 0x7fe1e0158100 2024-01-20T08:31:20.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.297+0000 7fe1dd7fa640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe1c406af20 0x7fe1c406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:20.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.298+0000 7fe1e4fb2640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe1c406af20 0x7fe1c406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:20.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.298+0000 7fe1e4fb2640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe1c406af20 0x7fe1c406d3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4036000 tx=0x7fe1d4030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:20.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.299+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe1d800ce30 con 0x7fe1e0158100 2024-01-20T08:31:20.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.309+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe1d8054920 con 0x7fe1e0158100 2024-01-20T08:31:20.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.485+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e00b08d0 con 0x7fe1e0158100 2024-01-20T08:31:20.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.561+0000 7fe1dd7fa640 1 -- 172.21.15.87:0/548343998 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fe1d8019020 con 0x7fe1e0158100 2024-01-20T08:31:20.565 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:20.565 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.563+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe1c406af20 msgr2=0x7fe1c406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.563+0000 7fe1e7216640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe1c406af20 0x7fe1c406d3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4036000 tx=0x7fe1d4030000 comp rx=0 tx=0).stop 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.563+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 msgr2=0x7fe1e006c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.563+0000 7fe1e7216640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 0x7fe1e006c650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b970 tx=0x7fe1d800be30 comp rx=0 tx=0).stop 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 shutdown_connections 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe1c406af20 0x7fe1c406d3e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e0158100 0x7fe1e006c650 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe1e0151740 0x7fe1e0071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 --2- 172.21.15.87:0/548343998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e00ca0e0 0x7fe1e006d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:20.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 >> 172.21.15.87:0/548343998 conn(0x7fe1e006a5b0 msgr2=0x7fe1e0155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:20.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 shutdown_connections 2024-01-20T08:31:20.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:20.564+0000 7fe1e7216640 1 -- 172.21.15.87:0/548343998 wait complete. 2024-01-20T08:31:21.181 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-20T08:31:09.352915+0000', 'modified': '2024-01-20T08:31:16.816824+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.87:6839/3149977016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 3149977016}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 3149977016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-01-20T08:31:21.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-20T08:31:21.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-20T08:31:21.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.283+0000 7f23704ce640 1 Processor -- start 2024-01-20T08:31:21.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.283+0000 7f23704ce640 1 -- start start 2024-01-20T08:31:21.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.284+0000 7f23704ce640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f23680c91e0 0x7f23680c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.284+0000 7f23704ce640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236815d8d0 con 0x7f23680c7ac0 2024-01-20T08:31:21.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.284+0000 7f23704ce640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680c3c80 con 0x7f2368153900 2024-01-20T08:31:21.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.284+0000 7f23704ce640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680c8070 con 0x7f23680c91e0 2024-01-20T08:31:21.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.285+0000 7f236da69640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f23680c91e0 0x7f23680c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.285+0000 7f236e26a640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2368153900 0x7f23680c99f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38486/0 (socket says 172.21.15.87:38486) 2024-01-20T08:31:21.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.285+0000 7f236e26a640 1 -- 172.21.15.87:0/752288252 learned_addr learned my addr 172.21.15.87:0/752288252 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:21.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.288+0000 7f236d268640 1 -- 172.21.15.87:0/752288252 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778708282 0 0) 0x7f23680c3c80 con 0x7f2368153900 2024-01-20T08:31:21.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.288+0000 7f236d268640 1 -- 172.21.15.87:0/752288252 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2354003520 con 0x7f2368153900 2024-01-20T08:31:21.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.288+0000 7f236da69640 1 -- 172.21.15.87:0/752288252 >> v1:172.21.15.116:6789/0 conn(0x7f2368153900 legacy=0x7f23680c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.288+0000 7f236da69640 1 -- 172.21.15.87:0/752288252 >> v1:172.21.15.87:6789/0 conn(0x7f23680c7ac0 legacy=0x7f2368159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.288+0000 7f236da69640 1 -- 172.21.15.87:0/752288252 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236815ccc0 con 0x7f23680c91e0 2024-01-20T08:31:21.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.293+0000 7f236da69640 1 --2- 172.21.15.87:0/752288252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c91e0 0x7f23680c3740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23580097a0 tx=0x7f235802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df27b5d7d39445e server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.297+0000 7f236d268640 1 -- 172.21.15.87:0/752288252 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235802f810 con 0x7f23680c91e0 2024-01-20T08:31:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.297+0000 7f236d268640 1 -- 172.21.15.87:0/752288252 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235802f970 con 0x7f23680c91e0 2024-01-20T08:31:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.297+0000 7f236d268640 1 -- 172.21.15.87:0/752288252 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235802fd00 con 0x7f23680c91e0 2024-01-20T08:31:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 -- 172.21.15.87:0/752288252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c91e0 msgr2=0x7f23680c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 --2- 172.21.15.87:0/752288252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c91e0 0x7f23680c3740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23580097a0 tx=0x7f235802b9f0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 -- 172.21.15.87:0/752288252 shutdown_connections 2024-01-20T08:31:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 --2- 172.21.15.87:0/752288252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c91e0 0x7f23680c3740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 -- 172.21.15.87:0/752288252 >> 172.21.15.87:0/752288252 conn(0x7f2368070de0 msgr2=0x7f23680711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 -- 172.21.15.87:0/752288252 shutdown_connections 2024-01-20T08:31:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.298+0000 7f23704ce640 1 -- 172.21.15.87:0/752288252 wait complete. 2024-01-20T08:31:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.299+0000 7f23704ce640 1 Processor -- start 2024-01-20T08:31:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.299+0000 7f23704ce640 1 -- start start 2024-01-20T08:31:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f23704ce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 0x7f23681548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f23704ce640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2368153900 0x7f2368145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f23704ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23681ef890 0x7f2368152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f23704ce640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23681f3590 con 0x7f23680c7ac0 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f23704ce640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368155e10 con 0x7f23681ef890 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f23704ce640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368156600 con 0x7f2368153900 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f236e26a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 0x7f23681548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23681ef890 0x7f2368152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.300+0000 7f236da69640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2368153900 0x7f2368145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.301+0000 7f236e26a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 0x7f23681548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38250/0 (socket says 172.21.15.87:38250) 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.301+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23681ef890 0x7f2368152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41162/0 (socket says 172.21.15.87:41162) 2024-01-20T08:31:21.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.301+0000 7f236e26a640 1 -- 172.21.15.87:0/2701473747 learned_addr learned my addr 172.21.15.87:0/2701473747 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.304+0000 7f236e26a640 1 -- 172.21.15.87:0/2701473747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2368153900 msgr2=0x7f2368145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.304+0000 7f236e26a640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2368153900 0x7f2368145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.304+0000 7f236e26a640 1 -- 172.21.15.87:0/2701473747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23681ef890 msgr2=0x7f2368152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.304+0000 7f236e26a640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23681ef890 0x7f2368152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.304+0000 7f236e26a640 1 -- 172.21.15.87:0/2701473747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009400 con 0x7f23680c7ac0 2024-01-20T08:31:21.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.306+0000 7f236e26a640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 0x7f23681548d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f236400caa0 tx=0x7f236400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.308+0000 7f235effd640 1 -- 172.21.15.87:0/2701473747 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400ac00 con 0x7f23680c7ac0 2024-01-20T08:31:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.308+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23681568c0 con 0x7f23680c7ac0 2024-01-20T08:31:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.308+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2368156e80 con 0x7f23680c7ac0 2024-01-20T08:31:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.309+0000 7f235effd640 1 -- 172.21.15.87:0/2701473747 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364001b60 con 0x7f23680c7ac0 2024-01-20T08:31:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.309+0000 7f235effd640 1 -- 172.21.15.87:0/2701473747 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236401c440 con 0x7f23680c7ac0 2024-01-20T08:31:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.311+0000 7f2373171740 1 Processor -- start 2024-01-20T08:31:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.311+0000 7f2373171740 1 -- start start 2024-01-20T08:31:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f2373171740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd663590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd662290 0x55eedd6a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 0x55eedd6a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f2373171740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd61e010 con 0x55eedd6a37f0 2024-01-20T08:31:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd5cfe60 con 0x55eedd6a3bd0 2024-01-20T08:31:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedd5cd550 con 0x55eedd662290 2024-01-20T08:31:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.312+0000 7f236da69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 0x55eedd6a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.313+0000 7f236da69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 0x55eedd6a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41164/0 (socket says 172.21.15.87:41164) 2024-01-20T08:31:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.313+0000 7f236da69640 1 -- 172.21.15.87:0/2798297179 learned_addr learned my addr 172.21.15.87:0/2798297179 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:21.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.314+0000 7f236e26a640 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd662290 0x55eedd6a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.315+0000 7f236da69640 1 -- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd662290 msgr2=0x55eedd6a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.315+0000 7f236da69640 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd662290 0x55eedd6a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.315+0000 7f236da69640 1 -- 172.21.15.87:0/2798297179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 msgr2=0x55eedd663590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.315+0000 7f236da69640 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd663590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.315+0000 7f236da69640 1 -- 172.21.15.87:0/2798297179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eedd5c0b00 con 0x55eedd6a3bd0 2024-01-20T08:31:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.317+0000 7f236da69640 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 0x55eedd6a2ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23580098c0 tx=0x7f2358030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee4bf533c07cc73 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2353fff640 1 -- 172.21.15.87:0/2798297179 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235803b040 con 0x55eedd6a3bd0 2024-01-20T08:31:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2353fff640 1 -- 172.21.15.87:0/2798297179 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235802fce0 con 0x55eedd6a3bd0 2024-01-20T08:31:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2353fff640 1 -- 172.21.15.87:0/2798297179 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358037640 con 0x55eedd6a3bd0 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 -- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 msgr2=0x55eedd6a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 0x55eedd6a2ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23580098c0 tx=0x7f2358030000 comp rx=0 tx=0).stop 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 -- 172.21.15.87:0/2798297179 shutdown_connections 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a3bd0 0x55eedd6a2ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd662290 0x55eedd6a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 --2- 172.21.15.87:0/2798297179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd663590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 -- 172.21.15.87:0/2798297179 >> 172.21.15.87:0/2798297179 conn(0x55eedd65f400 msgr2=0x55eedd662ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 -- 172.21.15.87:0/2798297179 shutdown_connections 2024-01-20T08:31:21.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.322+0000 7f2373171740 1 -- 172.21.15.87:0/2798297179 wait complete. 2024-01-20T08:31:21.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.323+0000 7f2373171740 1 -- start start 2024-01-20T08:31:21.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.323+0000 7f2373171740 1 Processor -- start 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 0x55eedd683720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a77a0 0x55eedd681320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd5c8e40 con 0x55eedd6a77a0 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd582250 con 0x55eedd6a37f0 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedd581ef0 con 0x55eedd6a3bd0 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.324+0000 7f2373171740 10 client.0 _ll_register_callbacks cb 0x55eedd586720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.325+0000 7f236e26a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.325+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 0x55eedd683720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.325+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 0x55eedd683720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41178/0 (socket says 172.21.15.87:41178) 2024-01-20T08:31:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.325+0000 7f236e26a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:55990/0 (socket says 172.21.15.87:55990) 2024-01-20T08:31:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.325+0000 7f236ea6b640 1 -- 172.21.15.87:0/4114137749 learned_addr learned my addr 172.21.15.87:0/4114137749 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.328+0000 7f236ea6b640 1 -- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 msgr2=0x55eedd6b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.328+0000 7f236ea6b640 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.328+0000 7f236ea6b640 1 -- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a77a0 msgr2=0x55eedd681320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.328+0000 7f236ea6b640 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a77a0 0x55eedd681320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.328+0000 7f236ea6b640 1 -- 172.21.15.87:0/4114137749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236815ccc0 con 0x55eedd6a37f0 2024-01-20T08:31:21.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.330+0000 7f235effd640 1 -- 172.21.15.87:0/2701473747 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f236401c980 con 0x7f23680c7ac0 2024-01-20T08:31:21.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.330+0000 7f235effd640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f234c06ae40 0x7f234c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.331+0000 7f236e26a640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f234c06ae40 0x7f234c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.331+0000 7f236ea6b640 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 0x55eedd683720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2360001eb0 tx=0x7f236000f660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afbe94af532f8ab1 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.331+0000 7f23517fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-01-20T08:31:21.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.331+0000 7f236e26a640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f234c06ae40 0x7f234c06d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2364055110 tx=0x7f2364007040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.334+0000 7f235effd640 1 -- 172.21.15.87:0/2701473747 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2364058c10 con 0x7f23680c7ac0 2024-01-20T08:31:21.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.334+0000 7f23517fa640 1 -- 172.21.15.87:0/4114137749 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600050d0 con 0x55eedd6a37f0 2024-01-20T08:31:21.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.334+0000 7f23517fa640 1 -- 172.21.15.87:0/4114137749 <== mon.1 v2:172.21.15.116:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2360005230 con 0x55eedd6a37f0 2024-01-20T08:31:21.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.334+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eedd5c1cd0 con 0x55eedd6a37f0 2024-01-20T08:31:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.336+0000 7f23517fa640 1 -- 172.21.15.87:0/4114137749 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23600053f0 con 0x55eedd6a37f0 2024-01-20T08:31:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.336+0000 7f2373171740 10 client.4384 fetch_fsmap learned FSMap version 7 2024-01-20T08:31:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.336+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55eedd5c74c0 con 0x55eedd6a37f0 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f23517fa640 1 -- 172.21.15.87:0/4114137749 <== mon.1 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eedd5c1cd0 con 0x55eedd6a37f0 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f23517fa640 1 -- 172.21.15.87:0/4114137749 <== mon.1 v2:172.21.15.116:3300/0 5 ==== fsmap(e 7) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f2360012070 con 0x55eedd6a37f0 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f2373171740 10 client.4384 fetch_fsmap finished waiting for FSMap version 7 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f2373171740 10 client.4384 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f2373171740 20 client.4384 populate_metadata read hostname 'smithi087' 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f2373171740 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd6ad9f0 0x55eedd6c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f2373171740 4 client.4384 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-20T08:31:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.341+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 --> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eedd5c19f0 con 0x55eedd6ad9f0 2024-01-20T08:31:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.349+0000 7f236da69640 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd6ad9f0 0x55eedd6c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.434+0000 7f23517fa640 10 client.4384 ms_handle_connect on v2:172.21.15.87:6838/3149977016 2024-01-20T08:31:21.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.434+0000 7f236da69640 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd6ad9f0 0x55eedd6c22e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2560441690abdba4 server_cookie=a694f70e617e4c1b in_seq=0 out_seq=0 2024-01-20T08:31:21.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f23517fa640 1 -- 172.21.15.87:0/4114137749 <== mds.0 v2:172.21.15.87:6838/3149977016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eedd5c19f0 con 0x55eedd6ad9f0 2024-01-20T08:31:21.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f23517fa640 10 client.4384 handle_command_reply: tid=0 2024-01-20T08:31:21.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 client.4384 shutdown 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 msgr2=0x55eedd683720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 0x55eedd683720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2360001eb0 tx=0x7f236000f660 comp rx=0 tx=0).stop 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 shutdown_connections 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd6ad9f0 0x55eedd6c22e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a77a0 0x55eedd681320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f23517fa640 0 client.4384 ms_handle_reset on v2:172.21.15.87:6838/3149977016 2024-01-20T08:31:21.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 --2- 172.21.15.87:0/4114137749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a37f0 0x55eedd683720 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 >> 172.21.15.87:0/4114137749 conn(0x55eedd662290 msgr2=0x55eedd661250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 shutdown_connections 2024-01-20T08:31:21.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 1 -- 172.21.15.87:0/4114137749 wait complete. 2024-01-20T08:31:21.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.554+0000 7f2373171740 20 client.4384 trim_cache size 0 max 16384 2024-01-20T08:31:21.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 Processor -- start 2024-01-20T08:31:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 -- start start 2024-01-20T08:31:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a37f0 0x55eedd680d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 0x55eedd688820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd688d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd547300 con 0x55eedd6a3bd0 2024-01-20T08:31:21.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd6abec0 con 0x55eedd6a77a0 2024-01-20T08:31:21.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.570+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedd663a40 con 0x55eedd6a37f0 2024-01-20T08:31:21.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.571+0000 7f236da69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 0x55eedd688820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.571+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a37f0 0x55eedd680d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.571+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd688d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.571+0000 7f236da69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 0x55eedd688820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38276/0 (socket says 172.21.15.87:38276) 2024-01-20T08:31:21.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.571+0000 7f236da69640 1 -- 172.21.15.87:0/211863193 learned_addr learned my addr 172.21.15.87:0/211863193 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:21.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.573+0000 7f236da69640 1 -- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a37f0 msgr2=0x55eedd680d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.573+0000 7f236da69640 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a37f0 0x55eedd680d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.573+0000 7f236da69640 1 -- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 msgr2=0x55eedd688d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.573+0000 7f236da69640 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd688d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.573+0000 7f236da69640 1 -- 172.21.15.87:0/211863193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eedd564150 con 0x55eedd6a3bd0 2024-01-20T08:31:21.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.573+0000 7f236ea6b640 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd688d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:21.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.575+0000 7f236da69640 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 0x55eedd688820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f235802c000 tx=0x7f235802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fed44a1a4b72c6b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.579+0000 7f23537fe640 1 -- 172.21.15.87:0/211863193 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235803b040 con 0x55eedd6a3bd0 2024-01-20T08:31:21.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.579+0000 7f23537fe640 1 -- 172.21.15.87:0/211863193 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358054070 con 0x55eedd6a3bd0 2024-01-20T08:31:21.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.579+0000 7f23537fe640 1 -- 172.21.15.87:0/211863193 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358004440 con 0x55eedd6a3bd0 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.579+0000 7f2373171740 1 -- 172.21.15.87:0/211863193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 msgr2=0x55eedd688820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.579+0000 7f2373171740 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 0x55eedd688820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f235802c000 tx=0x7f235802e000 comp rx=0 tx=0).stop 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 -- 172.21.15.87:0/211863193 shutdown_connections 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd688d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a3bd0 0x55eedd688820 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 --2- 172.21.15.87:0/211863193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a37f0 0x55eedd680d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 -- 172.21.15.87:0/211863193 >> 172.21.15.87:0/211863193 conn(0x55eedd662290 msgr2=0x55eedd660190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 -- 172.21.15.87:0/211863193 shutdown_connections 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 -- 172.21.15.87:0/211863193 wait complete. 2024-01-20T08:31:21.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 -- start start 2024-01-20T08:31:21.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.580+0000 7f2373171740 1 Processor -- start 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd6a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd7080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd6a28f0 con 0x55eedd6a37f0 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedd6c70a0 con 0x55eedd6a77a0 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedd65c670 con 0x55eedd6a3bd0 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f2373171740 10 client.0 _ll_register_callbacks cb 0x7f2360001220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.581+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd6a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.582+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd7080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.582+0000 7f236ea6b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd6a2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38292/0 (socket says 172.21.15.87:38292) 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.582+0000 7f236ea6b640 1 -- 172.21.15.87:0/2963731539 learned_addr learned my addr 172.21.15.87:0/2963731539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:21.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.582+0000 7f236da69640 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.585+0000 7f236ea6b640 1 -- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 msgr2=0x55eedd6c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.585+0000 7f236ea6b640 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.585+0000 7f236ea6b640 1 -- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 msgr2=0x55eedd6a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.585+0000 7f236ea6b640 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd6a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.585+0000 7f236ea6b640 1 -- 172.21.15.87:0/2963731539 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236002f040 con 0x55eedd6a77a0 2024-01-20T08:31:21.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.587+0000 7f236ea6b640 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd7080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2360001160 tx=0x7f2360005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5aba2c3580af34 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.587+0000 7f23517fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-01-20T08:31:21.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.591+0000 7f23517fa640 1 -- 172.21.15.87:0/2963731539 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360005370 con 0x55eedd6a77a0 2024-01-20T08:31:21.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.591+0000 7f23517fa640 1 -- 172.21.15.87:0/2963731539 <== mon.1 v2:172.21.15.116:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f236001b7c0 con 0x55eedd6a77a0 2024-01-20T08:31:21.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.591+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f235802fce0 con 0x55eedd6a77a0 2024-01-20T08:31:21.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.592+0000 7f23517fa640 1 -- 172.21.15.87:0/2963731539 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f236001b980 con 0x55eedd6a77a0 2024-01-20T08:31:21.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.592+0000 7f2373171740 10 client.4390 fetch_fsmap learned FSMap version 7 2024-01-20T08:31:21.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.592+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55eedd5c0b00 con 0x55eedd6a77a0 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.596+0000 7f23517fa640 1 -- 172.21.15.87:0/2963731539 <== mon.1 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235802fce0 con 0x55eedd6a77a0 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.596+0000 7f23517fa640 1 -- 172.21.15.87:0/2963731539 <== mon.1 v2:172.21.15.116:3300/0 5 ==== fsmap(e 7) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f2360033070 con 0x55eedd6a77a0 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.596+0000 7f2373171740 10 client.4390 fetch_fsmap finished waiting for FSMap version 7 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.597+0000 7f2373171740 10 client.4390 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.597+0000 7f2373171740 20 client.4390 populate_metadata read hostname 'smithi087' 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.597+0000 7f2373171740 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd7119f0 0x55eedd711dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.597+0000 7f2373171740 4 client.4390 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.597+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 --> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eedd2c86c0 con 0x55eedd7119f0 2024-01-20T08:31:21.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.597+0000 7f236da69640 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd7119f0 0x55eedd711dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.600+0000 7f236da69640 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd7119f0 0x55eedd711dd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b807c8bd2e740f server_cookie=660249792843d8db in_seq=0 out_seq=0 2024-01-20T08:31:21.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.600+0000 7f23517fa640 10 client.4390 ms_handle_connect on v2:172.21.15.87:6838/3149977016 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f23517fa640 1 -- 172.21.15.87:0/2963731539 <== mds.0 v2:172.21.15.87:6838/3149977016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+316 (crc 0 0 0) 0x55eedd2c86c0 con 0x55eedd7119f0 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f23517fa640 10 client.4390 handle_command_reply: tid=0 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 client.4390 shutdown 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 msgr2=0x55eedd7080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd7080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2360001160 tx=0x7f2360005000 comp rx=0 tx=0).stop 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 shutdown_connections 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eedd7119f0 0x55eedd711dd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eedd6a77a0 0x55eedd7080d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eedd6a3bd0 0x55eedd6c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f23517fa640 0 client.4390 ms_handle_reset on v2:172.21.15.87:6838/3149977016 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 --2- 172.21.15.87:0/2963731539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eedd6a37f0 0x55eedd6a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 >> 172.21.15.87:0/2963731539 conn(0x55eedd662290 msgr2=0x55eedd680d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 shutdown_connections 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.617+0000 7f2373171740 1 -- 172.21.15.87:0/2963731539 wait complete. 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f2373171740 20 client.4390 trim_cache size 0 max 16384 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "2b2378c9-6207-4be4-af26-3b0cbbc1772f", 2024-01-20T08:31:21.622 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4293, 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 10.999398757, 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 7, 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 25, 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 25, 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 21.2969106 2024-01-20T08:31:21.623 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f234c06ae40 msgr2=0x7f234c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f234c06ae40 0x7f234c06d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2364055110 tx=0x7f2364007040 comp rx=0 tx=0).stop 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 msgr2=0x7f23681548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 0x7f23681548d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f236400caa0 tx=0x7f236400cf60 comp rx=0 tx=0).stop 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 shutdown_connections 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f234c06ae40 0x7f234c06d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23681ef890 0x7f2368152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2368153900 0x7f2368145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 --2- 172.21.15.87:0/2701473747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23680c7ac0 0x7f23681548d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 >> 172.21.15.87:0/2701473747 conn(0x7f2368070de0 msgr2=0x7f236806edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 shutdown_connections 2024-01-20T08:31:21.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.618+0000 7f23704ce640 1 -- 172.21.15.87:0/2701473747 wait complete. 2024-01-20T08:31:21.632 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-20T08:31:21.632 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:21.632 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-20T08:31:21.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 Processor -- start 2024-01-20T08:31:21.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 -- start start 2024-01-20T08:31:21.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c815a500 0x7ff1c80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1c80cee30 0x7ff1c80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c80cadf0 con 0x7ff1c80c8d20 2024-01-20T08:31:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c80cb0d0 con 0x7ff1c80cee30 2024-01-20T08:31:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.729+0000 7ff1cda31640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80cb3b0 con 0x7ff1c815a500 2024-01-20T08:31:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.730+0000 7ff1c6ffd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff1c80c8d20 0x7ff1c80cc170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52198/0 (socket says 172.21.15.87:52198) 2024-01-20T08:31:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.730+0000 7ff1c77fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1c80cee30 0x7ff1c80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.730+0000 7ff1c6ffd640 1 -- 172.21.15.87:0/1235569529 learned_addr learned my addr 172.21.15.87:0/1235569529 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.730+0000 7ff1c67fc640 1 --2- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c815a500 0x7ff1c80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.732+0000 7ff1c77fe640 1 -- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c815a500 msgr2=0x7ff1c80c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:21.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.732+0000 7ff1c77fe640 1 --2- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c815a500 0x7ff1c80c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.732+0000 7ff1c77fe640 1 -- 172.21.15.87:0/1235569529 >> v1:172.21.15.87:6789/0 conn(0x7ff1c80c8d20 legacy=0x7ff1c80cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.732+0000 7ff1c77fe640 1 -- 172.21.15.87:0/1235569529 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80cb690 con 0x7ff1c80cee30 2024-01-20T08:31:21.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.734+0000 7ff1c77fe640 1 --2- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c80cf2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80097a0 tx=0x7ff1b802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9664194a31e2d4d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1c5ffb640 1 -- 172.21.15.87:0/1235569529 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8030610 con 0x7ff1c80cee30 2024-01-20T08:31:21.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1c5ffb640 1 -- 172.21.15.87:0/1235569529 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8030770 con 0x7ff1c80cee30 2024-01-20T08:31:21.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1c5ffb640 1 -- 172.21.15.87:0/1235569529 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8030b00 con 0x7ff1c80cee30 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 -- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 msgr2=0x7ff1c80cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 --2- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c80cf2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80097a0 tx=0x7ff1b802b960 comp rx=0 tx=0).stop 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 -- 172.21.15.87:0/1235569529 shutdown_connections 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 --2- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c80cf2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 --2- 172.21.15.87:0/1235569529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c815a500 0x7ff1c80c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 -- 172.21.15.87:0/1235569529 >> 172.21.15.87:0/1235569529 conn(0x7ff1c808af90 msgr2=0x7ff1c808b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 -- 172.21.15.87:0/1235569529 shutdown_connections 2024-01-20T08:31:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.738+0000 7ff1cda31640 1 -- 172.21.15.87:0/1235569529 wait complete. 2024-01-20T08:31:21.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.739+0000 7ff1cda31640 1 Processor -- start 2024-01-20T08:31:21.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 -- start start 2024-01-20T08:31:21.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c80c8d20 0x7ff1c80c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c81efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 0x7ff1c81eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81521a0 con 0x7ff1c815a500 2024-01-20T08:31:21.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81517d0 con 0x7ff1c80cee30 2024-01-20T08:31:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.740+0000 7ff1cda31640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8151ec0 con 0x7ff1c80c8d20 2024-01-20T08:31:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.741+0000 7ff1c77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 0x7ff1c81eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.741+0000 7ff1c67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c81efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.741+0000 7ff1c6ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c80c8d20 0x7ff1c80c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.741+0000 7ff1c77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 0x7ff1c81eff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38306/0 (socket says 172.21.15.87:38306) 2024-01-20T08:31:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.741+0000 7ff1c77fe640 1 -- 172.21.15.87:0/3790680586 learned_addr learned my addr 172.21.15.87:0/3790680586 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.745+0000 7ff1c77fe640 1 -- 172.21.15.87:0/3790680586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c80c8d20 msgr2=0x7ff1c80c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.745+0000 7ff1c77fe640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c80c8d20 0x7ff1c80c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.745+0000 7ff1c77fe640 1 -- 172.21.15.87:0/3790680586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 msgr2=0x7ff1c81efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.745+0000 7ff1c77fe640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c81efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.745+0000 7ff1c77fe640 1 -- 172.21.15.87:0/3790680586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009400 con 0x7ff1c815a500 2024-01-20T08:31:21.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.747+0000 7ff1c77fe640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 0x7ff1c81eff80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80098c0 tx=0x7ff1b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.751+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b803e040 con 0x7ff1c815a500 2024-01-20T08:31:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.752+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8039c10 con 0x7ff1c815a500 2024-01-20T08:31:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.752+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c8151210 con 0x7ff1c815a500 2024-01-20T08:31:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.752+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8030e10 con 0x7ff1c815a500 2024-01-20T08:31:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.752+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c81ef520 con 0x7ff1c815a500 2024-01-20T08:31:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.753+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c81ef380 con 0x7ff1c815a500 2024-01-20T08:31:21.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.772+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff1b804d3b0 con 0x7ff1c815a500 2024-01-20T08:31:21.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.772+0000 7ff1affff640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff1b406af20 0x7ff1b406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:21.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.772+0000 7ff1c6ffd640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff1b406af20 0x7ff1b406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:21.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:21.772+0000 7ff1c6ffd640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff1b406af20 0x7ff1b406d3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0007fd0 tx=0x7ff1c0007e80 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.004+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff1b807a9b0 con 0x7ff1c815a500 2024-01-20T08:31:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.004+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff1b8076cf0 con 0x7ff1c815a500 2024-01-20T08:31:22.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.214+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1c81ef380 con 0x7ff1c815a500 2024-01-20T08:31:22.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.221+0000 7ff1affff640 1 -- 172.21.15.87:0/3790680586 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff1c81ef380 con 0x7ff1c815a500 2024-01-20T08:31:22.224 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff1b406af20 msgr2=0x7ff1b406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff1b406af20 0x7ff1b406d3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0007fd0 tx=0x7ff1c0007e80 comp rx=0 tx=0).stop 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 msgr2=0x7ff1c81eff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 0x7ff1c81eff80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80098c0 tx=0x7ff1b8004730 comp rx=0 tx=0).stop 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 shutdown_connections 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff1b406af20 0x7ff1b406d3e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815a500 0x7ff1c81eff80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1c80cee30 0x7ff1c81efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 --2- 172.21.15.87:0/3790680586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c80c8d20 0x7ff1c80c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 >> 172.21.15.87:0/3790680586 conn(0x7ff1c808af90 msgr2=0x7ff1c80c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 shutdown_connections 2024-01-20T08:31:22.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:22.223+0000 7ff1cda31640 1 -- 172.21.15.87:0/3790680586 wait complete. 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-20T08:31:22.237 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-01-20T08:31:22.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.238 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-01-20T08:31:22.252 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-20T08:31:22.253 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-01-20T08:31:22.254 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-20T08:31:22.254 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-01-20T08:31:22.254 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-20T08:31:22.254 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-01-20T08:31:22.254 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-01-20T08:31:22.254 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-01-20T08:31:22.254 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-20T08:31:22.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-20T08:31:22.255 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:31:22.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.426 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-20T08:31:22.452 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-01-20T08:31:22.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.454 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-01-20T08:31:22.522 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-01-20T08:31:22.522 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-20T08:31:22.522 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-20T08:31:22.522 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-20T08:31:22.523 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-20T08:31:22.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.523 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:31:22.523 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:31:22.523 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-20T08:31:22.523 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-20T08:31:22.523 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-20T08:31:22.523 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:31:22.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.813 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-01-20T08:31:22.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.828 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-01-20T08:31:22.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.882 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:31:22.882 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:31:22.882 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:22.882 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-01-20T08:31:22.883 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:31:22.966 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-01-20T08:31:22.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-01-20T08:31:22.967 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:31:23.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:23.114 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:31:23.114 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:31:23.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-01-20T08:31:23.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-01-20T08:31:23.114 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:31:23.156 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-01-20T08:31:23.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-20T08:31:23.156 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:23.211 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-01-20T08:31:23.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-20T08:31:23.212 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:23.266 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:31:23.335 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:31:23.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:31:23.392 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:23.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:23.393 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:31:23.446 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-20T08:31:23.446 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-01-20T08:31:23.488 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:31:23.517 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:31:23.573 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:31:23.573 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:31:23.574 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:23.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:23.576 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:31:23.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:2024-01-20T08:31:23.602+0000 7efe073b8480 -1 init, newargv = 0x557989b20570 newargc=15 2024-01-20T08:31:23.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:ceph-fuse[43157]: starting ceph client 2024-01-20T08:31:23.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:ceph-fuse[43157]: starting fuse 2024-01-20T08:31:24.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:31:24.748 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:31:24.803 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:31:24.803 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:31:24.803 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:31:24.803 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:31:24.803 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:31:24.803 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:31:24.804 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:24.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:24.805 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:31:24.858 INFO:teuthology.orchestra.run.smithi087.stdout:41 2024-01-20T08:31:24.858 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-01-20T08:31:24.858 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-01-20T08:31:24.859 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-01-20T08:31:24.860 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-01-20T08:31:24.860 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-20T08:31:24.860 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:31:24.860 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.admin")) 2024-01-20T08:31:24.860 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-01-20T08:31:24.976 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.admin.43157.asok 2024-01-20T08:31:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-20T08:31:24.986 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43157.asok 2024-01-20T08:31:24.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:24.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43157.asok status 2024-01-20T08:31:25.120 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "a00c9940991c5e1e9ab595faa3f6e1281d8ae167", 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-763-ga00c9940 (a00c9940991c5e1e9ab595faa3f6e1281d8ae167) squid (dev)", 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "admin", 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "43157", 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4405, 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-01-20T08:31:25.121 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4405 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3224695415 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3224695415 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4405 172.21.15.87:0/3224695415", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "172.21.15.87:0/3224695415", 2024-01-20T08:31:25.122 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-01-20T08:31:25.123 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 7, 2024-01-20T08:31:25.123 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 25, 2024-01-20T08:31:25.123 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-01-20T08:31:25.123 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-01-20T08:31:25.123 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-01-20T08:31:25.123 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:31:25.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:25.130 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-01-20T08:31:25.130 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:25.155 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-01-20T08:31:25.155 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:25.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:25.155 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:25.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:25.314 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-01-20T08:31:25.517 INFO:teuthology.orchestra.run.smithi087.stdout:# file: . 2024-01-20T08:31:25.517 INFO:teuthology.orchestra.run.smithi087.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-01-20T08:31:25.517 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:25.517 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:31:25.517 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-20T08:31:25.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:25.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.668+0000 7f3ea7207640 1 Processor -- start 2024-01-20T08:31:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.669+0000 7f3ea7207640 1 -- start start 2024-01-20T08:31:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.669+0000 7f3ea7207640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea00c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.669+0000 7f3ea7207640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ea0156ea0 0x7f3ea01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.669+0000 7f3ea7207640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea0071910 con 0x7f3ea0150350 2024-01-20T08:31:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.669+0000 7f3ea7207640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea0150730 con 0x7f3ea0156ea0 2024-01-20T08:31:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.669+0000 7f3ea7207640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0071b60 con 0x7f3ea00bff90 2024-01-20T08:31:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.670+0000 7f3ea57a4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ea0156ea0 0x7f3ea01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.670+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea00c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.670+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea00c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56678/0 (socket says 172.21.15.87:56678) 2024-01-20T08:31:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.670+0000 7f3e9ffff640 1 -- 172.21.15.87:0/435409614 learned_addr learned my addr 172.21.15.87:0/435409614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.672+0000 7f3e9ffff640 1 -- 172.21.15.87:0/435409614 >> v2:172.21.15.87:3300/0 conn(0x7f3ea0156ea0 msgr2=0x7f3ea01598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.672+0000 7f3e9ffff640 1 --2- 172.21.15.87:0/435409614 >> v2:172.21.15.87:3300/0 conn(0x7f3ea0156ea0 0x7f3ea01598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.672+0000 7f3e9ffff640 1 -- 172.21.15.87:0/435409614 >> v1:172.21.15.87:6789/0 conn(0x7f3ea0150350 legacy=0x7f3ea00cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.672+0000 7f3e9ffff640 1 -- 172.21.15.87:0/435409614 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea01563c0 con 0x7f3ea00bff90 2024-01-20T08:31:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.674+0000 7f3e9ffff640 1 --2- 172.21.15.87:0/435409614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea00c57c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009870 tx=0x7f3e9402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a5886338a4bc74 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.678+0000 7f3e9f7fe640 1 -- 172.21.15.87:0/435409614 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94030840 con 0x7f3ea00bff90 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.678+0000 7f3e9f7fe640 1 -- 172.21.15.87:0/435409614 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940309a0 con 0x7f3ea00bff90 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.678+0000 7f3e9f7fe640 1 -- 172.21.15.87:0/435409614 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94030d30 con 0x7f3ea00bff90 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 -- 172.21.15.87:0/435409614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 msgr2=0x7f3ea00c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 --2- 172.21.15.87:0/435409614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea00c57c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009870 tx=0x7f3e9402b980 comp rx=0 tx=0).stop 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 -- 172.21.15.87:0/435409614 shutdown_connections 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 --2- 172.21.15.87:0/435409614 >> v2:172.21.15.87:3300/0 conn(0x7f3ea0156ea0 0x7f3ea01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 --2- 172.21.15.87:0/435409614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea00c57c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 -- 172.21.15.87:0/435409614 >> 172.21.15.87:0/435409614 conn(0x7f3ea006d510 msgr2=0x7f3ea00663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 -- 172.21.15.87:0/435409614 shutdown_connections 2024-01-20T08:31:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.679+0000 7f3ea7207640 1 -- 172.21.15.87:0/435409614 wait complete. 2024-01-20T08:31:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.680+0000 7f3ea7207640 1 Processor -- start 2024-01-20T08:31:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.680+0000 7f3ea7207640 1 -- start start 2024-01-20T08:31:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea7207640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea0146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea7207640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea0150350 0x7f3ea00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea7207640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ea0156ea0 0x7f3ea0144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea7207640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01f09d0 con 0x7f3ea0150350 2024-01-20T08:31:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea7207640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01f0f00 con 0x7f3ea0156ea0 2024-01-20T08:31:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea7207640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0144a50 con 0x7f3ea00bff90 2024-01-20T08:31:25.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea4fa3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea0146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:25.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea0150350 0x7f3ea00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:25.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.681+0000 7f3ea57a4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ea0156ea0 0x7f3ea0144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:25.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.682+0000 7f3ea4fa3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea0146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56694/0 (socket says 172.21.15.87:56694) 2024-01-20T08:31:25.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.682+0000 7f3ea4fa3640 1 -- 172.21.15.87:0/4021887170 learned_addr learned my addr 172.21.15.87:0/4021887170 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.685+0000 7f3e9ffff640 1 -- 172.21.15.87:0/4021887170 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 msgr2=0x7f3ea0146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.685+0000 7f3e9ffff640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea0146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.685+0000 7f3e9ffff640 1 -- 172.21.15.87:0/4021887170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ea0156ea0 msgr2=0x7f3ea0144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.685+0000 7f3e9ffff640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ea0156ea0 0x7f3ea0144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.685+0000 7f3e9ffff640 1 -- 172.21.15.87:0/4021887170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940094d0 con 0x7f3ea0150350 2024-01-20T08:31:25.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.687+0000 7f3e9ffff640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea0150350 0x7f3ea00710e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e94033000 tx=0x7f3e94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:25.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.691+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94047070 con 0x7f3ea0150350 2024-01-20T08:31:25.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.691+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94039a90 con 0x7f3ea0150350 2024-01-20T08:31:25.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.691+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea01f0c20 con 0x7f3ea0150350 2024-01-20T08:31:25.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.691+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94039e60 con 0x7f3ea0150350 2024-01-20T08:31:25.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.691+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea014d540 con 0x7f3ea0150350 2024-01-20T08:31:25.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.692+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea0144c10 con 0x7f3ea0150350 2024-01-20T08:31:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.710+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3e9404c410 con 0x7f3ea0150350 2024-01-20T08:31:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.710+0000 7f3e9d7fa640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3e8c06aed0 0x7f3e8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.710+0000 7f3ea4fa3640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3e8c06aed0 0x7f3e8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.711+0000 7f3ea4fa3640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3e8c06aed0 0x7f3e8c06d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007fd0 tx=0x7f3e98009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:25.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.713+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3e9403e070 con 0x7f3ea0150350 2024-01-20T08:31:25.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:25.945+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3e94077540 con 0x7f3ea0150350 2024-01-20T08:31:26.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.118+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ea0074290 con 0x7f3ea0150350 2024-01-20T08:31:26.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.163+0000 7f3e9d7fa640 1 -- 172.21.15.87:0/4021887170 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f3e9407ab60 con 0x7f3ea0150350 2024-01-20T08:31:26.167 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:26.167 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.165+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3e8c06aed0 msgr2=0x7f3e8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.165+0000 7f3ea7207640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3e8c06aed0 0x7f3e8c06d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007fd0 tx=0x7f3e98009040 comp rx=0 tx=0).stop 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.165+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea0150350 msgr2=0x7f3ea00710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.165+0000 7f3ea7207640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea0150350 0x7f3ea00710e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e94033000 tx=0x7f3e94004730 comp rx=0 tx=0).stop 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 shutdown_connections 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3e8c06aed0 0x7f3e8c06d390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ea0156ea0 0x7f3ea0144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea0150350 0x7f3ea00710e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 --2- 172.21.15.87:0/4021887170 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ea00bff90 0x7f3ea0146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 >> 172.21.15.87:0/4021887170 conn(0x7f3ea006d510 msgr2=0x7f3ea00663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 shutdown_connections 2024-01-20T08:31:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.166+0000 7f3ea7207640 1 -- 172.21.15.87:0/4021887170 wait complete. 2024-01-20T08:31:26.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-20T08:31:06.500533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-20T08:31:08.175669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-20T08:31:13.903374+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-20T08:31:26.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:31:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.282+0000 7ff5c7bad640 1 Processor -- start 2024-01-20T08:31:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.282+0000 7ff5c7bad640 1 -- start start 2024-01-20T08:31:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.283+0000 7ff5c7bad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5c01517f0 0x7ff5c00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.283+0000 7ff5c7bad640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c00c9550 0x7ff5c00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.283+0000 7ff5c7bad640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00ca070 con 0x7ff5c00c90e0 2024-01-20T08:31:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.283+0000 7ff5c7bad640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00b0520 con 0x7ff5c01517f0 2024-01-20T08:31:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.283+0000 7ff5c7bad640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c0151bd0 con 0x7ff5c00c9550 2024-01-20T08:31:26.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.316+0000 7ff5c5949640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5c01517f0 0x7ff5c00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.316+0000 7ff5c5949640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5c01517f0 0x7ff5c00b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49880/0 (socket says 172.21.15.87:49880) 2024-01-20T08:31:26.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.316+0000 7ff5c5949640 1 -- 172.21.15.87:0/4006375369 learned_addr learned my addr 172.21.15.87:0/4006375369 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.318+0000 7ff5c5949640 1 -- 172.21.15.87:0/4006375369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c00c9550 msgr2=0x7ff5c00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.318+0000 7ff5c5949640 1 --2- 172.21.15.87:0/4006375369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c00c9550 0x7ff5c00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.318+0000 7ff5c5949640 1 -- 172.21.15.87:0/4006375369 >> v1:172.21.15.116:6789/0 conn(0x7ff5c00c90e0 legacy=0x7ff5c0157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:26.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.318+0000 7ff5c5949640 1 -- 172.21.15.87:0/4006375369 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0157ef0 con 0x7ff5c01517f0 2024-01-20T08:31:26.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.320+0000 7ff5c5949640 1 --2- 172.21.15.87:0/4006375369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c01517f0 0x7ff5c00b0dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00098a0 tx=0x7ff5b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66d36353e522658 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:26.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c4947640 1 -- 172.21.15.87:0/4006375369 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0031640 con 0x7ff5c01517f0 2024-01-20T08:31:26.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c4947640 1 -- 172.21.15.87:0/4006375369 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b00317a0 con 0x7ff5c01517f0 2024-01-20T08:31:26.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c4947640 1 -- 172.21.15.87:0/4006375369 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0031b30 con 0x7ff5c01517f0 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c7bad640 1 -- 172.21.15.87:0/4006375369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c01517f0 msgr2=0x7ff5c00b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/4006375369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c01517f0 0x7ff5c00b0dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00098a0 tx=0x7ff5b002b8f0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c7bad640 1 -- 172.21.15.87:0/4006375369 shutdown_connections 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/4006375369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c00c9550 0x7ff5c00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/4006375369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c01517f0 0x7ff5c00b0dd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.324+0000 7ff5c7bad640 1 -- 172.21.15.87:0/4006375369 >> 172.21.15.87:0/4006375369 conn(0x7ff5c006a5b0 msgr2=0x7ff5c006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.325+0000 7ff5c7bad640 1 -- 172.21.15.87:0/4006375369 shutdown_connections 2024-01-20T08:31:26.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.325+0000 7ff5c7bad640 1 -- 172.21.15.87:0/4006375369 wait complete. 2024-01-20T08:31:26.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.326+0000 7ff5c7bad640 1 Processor -- start 2024-01-20T08:31:26.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.326+0000 7ff5c7bad640 1 -- start start 2024-01-20T08:31:26.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c7bad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 0x7ff5c01487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c7bad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c00c9550 0x7ff5c01432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c7bad640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c01517f0 0x7ff5c014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c7bad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01492c0 con 0x7ff5c00c90e0 2024-01-20T08:31:26.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c7bad640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c014aba0 con 0x7ff5c00c9550 2024-01-20T08:31:26.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c7bad640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c014d780 con 0x7ff5c01517f0 2024-01-20T08:31:26.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.327+0000 7ff5c5949640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 0x7ff5c01487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.328+0000 7ff5c5949640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 0x7ff5c01487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49888/0 (socket says 172.21.15.87:49888) 2024-01-20T08:31:26.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.328+0000 7ff5c5949640 1 -- 172.21.15.87:0/2610978467 learned_addr learned my addr 172.21.15.87:0/2610978467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:26.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.331+0000 7ff5c5949640 1 -- 172.21.15.87:0/2610978467 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c01517f0 msgr2=0x7ff5c014fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.331+0000 7ff5c5949640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c01517f0 0x7ff5c014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.331+0000 7ff5c5949640 1 -- 172.21.15.87:0/2610978467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c00c9550 msgr2=0x7ff5c01432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.331+0000 7ff5c5949640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c00c9550 0x7ff5c01432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.331+0000 7ff5c5949640 1 -- 172.21.15.87:0/2610978467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00094d0 con 0x7ff5c00c90e0 2024-01-20T08:31:26.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.333+0000 7ff5c5949640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 0x7ff5c01487f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000c00 tx=0x7ff5b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:26.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.338+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b003ac60 con 0x7ff5c00c90e0 2024-01-20T08:31:26.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.338+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b003adc0 con 0x7ff5c00c90e0 2024-01-20T08:31:26.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.338+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c0148fe0 con 0x7ff5c00c90e0 2024-01-20T08:31:26.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.338+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0044630 con 0x7ff5c00c90e0 2024-01-20T08:31:26.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.338+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c01edb80 con 0x7ff5c00c90e0 2024-01-20T08:31:26.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.339+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c0074290 con 0x7ff5c00c90e0 2024-01-20T08:31:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.601+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff5b002f050 con 0x7ff5c00c90e0 2024-01-20T08:31:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.601+0000 7ff5b67fc640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff5a406af20 0x7ff5a406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.602+0000 7ff5c5148640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff5a406af20 0x7ff5a406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.602+0000 7ff5c5148640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff5a406af20 0x7ff5a406d3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5c014b0d0 tx=0x7ff5bc006ba0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:26.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.615+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff5b007ac20 con 0x7ff5c00c90e0 2024-01-20T08:31:26.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.616+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff5b0076860 con 0x7ff5c00c90e0 2024-01-20T08:31:26.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.785+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5c01440b0 con 0x7ff5c00c90e0 2024-01-20T08:31:26.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.790+0000 7ff5b67fc640 1 -- 172.21.15.87:0/2610978467 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5b0031e60 con 0x7ff5c00c90e0 2024-01-20T08:31:26.793 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:31:26.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff5a406af20 msgr2=0x7ff5a406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff5a406af20 0x7ff5a406d3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5c014b0d0 tx=0x7ff5bc006ba0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 msgr2=0x7ff5c01487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 0x7ff5c01487f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000c00 tx=0x7ff5b00046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 shutdown_connections 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff5a406af20 0x7ff5a406d3e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c01517f0 0x7ff5c014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c00c9550 0x7ff5c01432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 --2- 172.21.15.87:0/2610978467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c00c90e0 0x7ff5c01487f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 >> 172.21.15.87:0/2610978467 conn(0x7ff5c006a5b0 msgr2=0x7ff5c0156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 shutdown_connections 2024-01-20T08:31:26.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.792+0000 7ff5c7bad640 1 -- 172.21.15.87:0/2610978467 wait complete. 2024-01-20T08:31:26.805 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-20T08:31:26.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.905+0000 7f76cd282640 1 Processor -- start 2024-01-20T08:31:26.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.905+0000 7f76cd282640 1 -- start start 2024-01-20T08:31:26.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.905+0000 7f76cd282640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76c815d940 0x7f76c80c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76cd282640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c80cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76cd282640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c815a810 con 0x7f76c80c9120 2024-01-20T08:31:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76cd282640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c80b4400 con 0x7f76c815d940 2024-01-20T08:31:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76cd282640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80ca180 con 0x7f76c80ca8b0 2024-01-20T08:31:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76c77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c80cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76c6ffd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f76c80c9120 0x7f76c80cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:43330/0 (socket says 172.21.15.87:43330) 2024-01-20T08:31:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76c6ffd640 1 -- 172.21.15.87:0/1409996442 learned_addr learned my addr 172.21.15.87:0/1409996442 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:26.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.906+0000 7f76bffff640 1 --2- 172.21.15.87:0/1409996442 >> v2:172.21.15.87:3300/0 conn(0x7f76c815d940 0x7f76c80c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.908+0000 7f76c67fc640 1 -- 172.21.15.87:0/1409996442 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892130545 0 0) 0x7f76c815a810 con 0x7f76c80c9120 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.909+0000 7f76c67fc640 1 -- 172.21.15.87:0/1409996442 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b4003520 con 0x7f76c80c9120 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.909+0000 7f76c77fe640 1 -- 172.21.15.87:0/1409996442 >> v2:172.21.15.87:3300/0 conn(0x7f76c815d940 msgr2=0x7f76c80c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.909+0000 7f76c77fe640 1 --2- 172.21.15.87:0/1409996442 >> v2:172.21.15.87:3300/0 conn(0x7f76c815d940 0x7f76c80c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.909+0000 7f76c77fe640 1 -- 172.21.15.87:0/1409996442 >> v1:172.21.15.116:6789/0 conn(0x7f76c80c9120 legacy=0x7f76c80cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.909+0000 7f76c77fe640 1 -- 172.21.15.87:0/1409996442 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8154100 con 0x7f76c80ca8b0 2024-01-20T08:31:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.909+0000 7f76bffff640 1 --2- 172.21.15.87:0/1409996442 >> v2:172.21.15.87:3300/0 conn(0x7f76c815d940 0x7f76c80c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:26.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.911+0000 7f76c77fe640 1 --2- 172.21.15.87:0/1409996442 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c80cad20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009d90 tx=0x7f76c002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf9c9f35e8fce76 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.914+0000 7f76c67fc640 1 -- 172.21.15.87:0/1409996442 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c002cc50 con 0x7f76c80ca8b0 2024-01-20T08:31:26.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.914+0000 7f76c67fc640 1 -- 172.21.15.87:0/1409996442 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c002cdb0 con 0x7f76c80ca8b0 2024-01-20T08:31:26.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.914+0000 7f76c67fc640 1 -- 172.21.15.87:0/1409996442 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0034660 con 0x7f76c80ca8b0 2024-01-20T08:31:26.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 -- 172.21.15.87:0/1409996442 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 msgr2=0x7f76c80cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 --2- 172.21.15.87:0/1409996442 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c80cad20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009d90 tx=0x7f76c002bea0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 -- 172.21.15.87:0/1409996442 shutdown_connections 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 --2- 172.21.15.87:0/1409996442 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c80cad20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 --2- 172.21.15.87:0/1409996442 >> v2:172.21.15.87:3300/0 conn(0x7f76c815d940 0x7f76c80c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 -- 172.21.15.87:0/1409996442 >> 172.21.15.87:0/1409996442 conn(0x7f76c808af90 msgr2=0x7f76c808b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 -- 172.21.15.87:0/1409996442 shutdown_connections 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.915+0000 7f76cd282640 1 -- 172.21.15.87:0/1409996442 wait complete. 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.916+0000 7f76cd282640 1 Processor -- start 2024-01-20T08:31:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.916+0000 7f76cd282640 1 -- start start 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76cd282640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 0x7f76c8154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76cd282640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c81ef660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76cd282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76c815d940 0x7f76c80c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76cd282640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c8155220 con 0x7f76c80c9120 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76cd282640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81554f0 con 0x7f76c815d940 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76cd282640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80c4150 con 0x7f76c80ca8b0 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.917+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 0x7f76c8154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.918+0000 7f76bffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c81ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.918+0000 7f76c77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76c815d940 0x7f76c80c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.918+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 0x7f76c8154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49900/0 (socket says 172.21.15.87:49900) 2024-01-20T08:31:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.918+0000 7f76c6ffd640 1 -- 172.21.15.87:0/585803423 learned_addr learned my addr 172.21.15.87:0/585803423 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:26.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.921+0000 7f76c6ffd640 1 -- 172.21.15.87:0/585803423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 msgr2=0x7f76c81ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.921+0000 7f76c6ffd640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c81ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.921+0000 7f76c6ffd640 1 -- 172.21.15.87:0/585803423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76c815d940 msgr2=0x7f76c80c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:26.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.921+0000 7f76c6ffd640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76c815d940 0x7f76c80c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:26.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.921+0000 7f76c6ffd640 1 -- 172.21.15.87:0/585803423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009400 con 0x7f76c80c9120 2024-01-20T08:31:26.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.923+0000 7f76c6ffd640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 0x7f76c8154ce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76b800c460 tx=0x7f76b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:26.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.928+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8005ce0 con 0x7f76c80c9120 2024-01-20T08:31:26.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.928+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b8005e40 con 0x7f76c80c9120 2024-01-20T08:31:26.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.928+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8007290 con 0x7f76c80c9120 2024-01-20T08:31:26.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.928+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c0009a70 con 0x7f76c80c9120 2024-01-20T08:31:26.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.929+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c80c3e90 con 0x7f76c80c9120 2024-01-20T08:31:26.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.930+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c806d740 con 0x7f76c80c9120 2024-01-20T08:31:26.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.949+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f76b80073f0 con 0x7f76c80c9120 2024-01-20T08:31:26.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.950+0000 7f76bf7fe640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f76a806af50 0x7f76a806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:26.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.950+0000 7f76bffff640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f76a806af50 0x7f76a806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:26.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:26.950+0000 7f76bffff640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f76a806af50 0x7f76a806d410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76c815d780 tx=0x7f76b0009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:27.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.183+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f76b8057bd0 con 0x7f76c80c9120 2024-01-20T08:31:27.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.184+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f76b8053f10 con 0x7f76c80c9120 2024-01-20T08:31:27.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.359+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76c8074290 con 0x7f76c80c9120 2024-01-20T08:31:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.398+0000 7f76bf7fe640 1 -- 172.21.15.87:0/585803423 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76b8057530 con 0x7f76c80c9120 2024-01-20T08:31:27.401 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.400+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f76a806af50 msgr2=0x7f76a806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.400+0000 7f76cd282640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f76a806af50 0x7f76a806d410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76c815d780 tx=0x7f76b0009190 comp rx=0 tx=0).stop 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.400+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 msgr2=0x7f76c8154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.400+0000 7f76cd282640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 0x7f76c8154ce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76b800c460 tx=0x7f76b800c920 comp rx=0 tx=0).stop 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 shutdown_connections 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f76a806af50 0x7f76a806d410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76c815d940 0x7f76c80c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76c80ca8b0 0x7f76c81ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 --2- 172.21.15.87:0/585803423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c80c9120 0x7f76c8154ce0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 >> 172.21.15.87:0/585803423 conn(0x7f76c808af90 msgr2=0x7f76c80cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 shutdown_connections 2024-01-20T08:31:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.401+0000 7f76cd282640 1 -- 172.21.15.87:0/585803423 wait complete. 2024-01-20T08:31:27.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-20T08:31:27.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.514+0000 7fe717dde640 1 Processor -- start 2024-01-20T08:31:27.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.514+0000 7fe717dde640 1 -- start start 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe717dde640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe7101513a0 0x7fe7100663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe717dde640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe71006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe717dde640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100668f0 con 0x7fe71006bbc0 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe717dde640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe710158470 con 0x7fe7101513a0 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe717dde640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe710157530 con 0x7fe710151780 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe71637b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe71006bbc0 0x7fe710071990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36780/0 (socket says 172.21.15.87:36780) 2024-01-20T08:31:27.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe715b7a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe7101513a0 0x7fe7100663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:27.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe715379640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe71006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:27.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.515+0000 7fe71637b640 1 -- 172.21.15.87:0/896756571 learned_addr learned my addr 172.21.15.87:0/896756571 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.518+0000 7fe715b7a640 1 -- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 msgr2=0x7fe71006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.518+0000 7fe715b7a640 1 --2- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe71006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.518+0000 7fe715b7a640 1 -- 172.21.15.87:0/896756571 >> v1:172.21.15.87:6789/0 conn(0x7fe71006bbc0 legacy=0x7fe710071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.518+0000 7fe715b7a640 1 -- 172.21.15.87:0/896756571 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7101577f0 con 0x7fe7101513a0 2024-01-20T08:31:27.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.520+0000 7fe715b7a640 1 --2- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7101513a0 0x7fe7100663b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe708009d90 tx=0x7fe70802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c54835f4faa3398 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:27.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe714b78640 1 -- 172.21.15.87:0/896756571 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70802cb30 con 0x7fe7101513a0 2024-01-20T08:31:27.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe714b78640 1 -- 172.21.15.87:0/896756571 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70802cc90 con 0x7fe7101513a0 2024-01-20T08:31:27.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe714b78640 1 -- 172.21.15.87:0/896756571 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708034440 con 0x7fe7101513a0 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe717dde640 1 -- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7101513a0 msgr2=0x7fe7100663b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe717dde640 1 --2- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7101513a0 0x7fe7100663b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe708009d90 tx=0x7fe70802bd80 comp rx=0 tx=0).stop 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe717dde640 1 -- 172.21.15.87:0/896756571 shutdown_connections 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe717dde640 1 --2- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe71006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe717dde640 1 --2- 172.21.15.87:0/896756571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7101513a0 0x7fe7100663b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.524+0000 7fe717dde640 1 -- 172.21.15.87:0/896756571 >> 172.21.15.87:0/896756571 conn(0x7fe7100cb310 msgr2=0x7fe7100cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.525+0000 7fe717dde640 1 -- 172.21.15.87:0/896756571 shutdown_connections 2024-01-20T08:31:27.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.525+0000 7fe717dde640 1 -- 172.21.15.87:0/896756571 wait complete. 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 Processor -- start 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 -- start start 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe71006bbc0 0x7fe7100bb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7101513a0 0x7fe7100bf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe7101ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100bbb70 con 0x7fe7101513a0 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101eeea0 con 0x7fe71006bbc0 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.526+0000 7fe717dde640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101500c0 con 0x7fe710151780 2024-01-20T08:31:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.527+0000 7fe715379640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7101513a0 0x7fe7100bf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.527+0000 7fe71637b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe7101ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.527+0000 7fe715b7a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe71006bbc0 0x7fe7100bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.527+0000 7fe71637b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe7101ef2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56760/0 (socket says 172.21.15.87:56760) 2024-01-20T08:31:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.527+0000 7fe71637b640 1 -- 172.21.15.87:0/3112368779 learned_addr learned my addr 172.21.15.87:0/3112368779 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:27.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.531+0000 7fe715379640 1 -- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 msgr2=0x7fe7101ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.531+0000 7fe715379640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe7101ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.531+0000 7fe715379640 1 -- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe71006bbc0 msgr2=0x7fe7100bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.531+0000 7fe715379640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe71006bbc0 0x7fe7100bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.531+0000 7fe715379640 1 -- 172.21.15.87:0/3112368779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708009a70 con 0x7fe7101513a0 2024-01-20T08:31:27.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.531+0000 7fe715b7a640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe71006bbc0 0x7fe7100bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:27.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.533+0000 7fe715379640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7101513a0 0x7fe7100bf9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe70000e8a0 tx=0x7fe70000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:27.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.536+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700009e60 con 0x7fe7101513a0 2024-01-20T08:31:27.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.536+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70000c940 con 0x7fe7101513a0 2024-01-20T08:31:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.536+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70000ccd0 con 0x7fe7101513a0 2024-01-20T08:31:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.536+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100c4440 con 0x7fe7101513a0 2024-01-20T08:31:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.537+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe710156df0 con 0x7fe7101513a0 2024-01-20T08:31:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.537+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7101ef010 con 0x7fe7101513a0 2024-01-20T08:31:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.782+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe700013470 con 0x7fe7101513a0 2024-01-20T08:31:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.782+0000 7fe7067fc640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe6f406af20 0x7fe6f406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.783+0000 7fe715b7a640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe6f406af20 0x7fe6f406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.783+0000 7fe715b7a640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe6f406af20 0x7fe6f406d3e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe708036000 tx=0x7fe708030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:27.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.796+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe700058840 con 0x7fe7101513a0 2024-01-20T08:31:27.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.796+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe700054b80 con 0x7fe7101513a0 2024-01-20T08:31:27.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.967+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7101ef010 con 0x7fe7101513a0 2024-01-20T08:31:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.972+0000 7fe7067fc640 1 -- 172.21.15.87:0/3112368779 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7101ef010 con 0x7fe7101513a0 2024-01-20T08:31:27.975 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-01-20T08:31:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe6f406af20 msgr2=0x7fe6f406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe6f406af20 0x7fe6f406d3e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe708036000 tx=0x7fe708030000 comp rx=0 tx=0).stop 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7101513a0 msgr2=0x7fe7100bf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7101513a0 0x7fe7100bf9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe70000e8a0 tx=0x7fe70000ed60 comp rx=0 tx=0).stop 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 shutdown_connections 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe6f406af20 0x7fe6f406d3e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe710151780 0x7fe7101ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7101513a0 0x7fe7100bf9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 --2- 172.21.15.87:0/3112368779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe71006bbc0 0x7fe7100bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 >> 172.21.15.87:0/3112368779 conn(0x7fe7100cb310 msgr2=0x7fe71006f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 shutdown_connections 2024-01-20T08:31:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:27.974+0000 7fe717dde640 1 -- 172.21.15.87:0/3112368779 wait complete. 2024-01-20T08:31:27.988 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-20T08:31:28.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 Processor -- start 2024-01-20T08:31:28.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 -- start start 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f00153af0 0x7f1f000c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f0015d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0006bf00 con 0x7f1f0006c390 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f000c3e10 con 0x7f1f00153af0 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.090+0000 7f1f0565d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0015a140 con 0x7f1f0006c7b0 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.091+0000 7f1efe7fc640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1f0006c390 0x7f1f0015cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36786/0 (socket says 172.21.15.87:36786) 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.091+0000 7f1efe7fc640 1 -- 172.21.15.87:0/2748649637 learned_addr learned my addr 172.21.15.87:0/2748649637 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.091+0000 7f1eff7fe640 1 --2- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f0015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.091+0000 7f1efeffd640 1 --2- 172.21.15.87:0/2748649637 >> v2:172.21.15.116:3300/0 conn(0x7f1f00153af0 0x7f1f000c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.093+0000 7f1efdffb640 1 -- 172.21.15.87:0/2748649637 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256555669 0 0) 0x7f1f0006bf00 con 0x7f1f0006c390 2024-01-20T08:31:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.093+0000 7f1efdffb640 1 -- 172.21.15.87:0/2748649637 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee8003520 con 0x7f1f0006c390 2024-01-20T08:31:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.093+0000 7f1efeffd640 1 -- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 msgr2=0x7f1f0015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.094+0000 7f1efeffd640 1 --2- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f0015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.094+0000 7f1efeffd640 1 -- 172.21.15.87:0/2748649637 >> v1:172.21.15.87:6789/0 conn(0x7f1f0006c390 legacy=0x7f1f0015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.094+0000 7f1efeffd640 1 -- 172.21.15.87:0/2748649637 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000cb140 con 0x7f1f00153af0 2024-01-20T08:31:28.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.095+0000 7f1efeffd640 1 --2- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f00153af0 0x7f1f000c98d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00097a0 tx=0x7f1ef002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc7404cb1800f7f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:28.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.097+0000 7f1efdffb640 1 -- 172.21.15.87:0/2748649637 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0030610 con 0x7f1f00153af0 2024-01-20T08:31:28.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.098+0000 7f1efdffb640 1 -- 172.21.15.87:0/2748649637 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0030770 con 0x7f1f00153af0 2024-01-20T08:31:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 -- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f00153af0 msgr2=0x7f1f000c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 --2- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f00153af0 0x7f1f000c98d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00097a0 tx=0x7f1ef002b960 comp rx=0 tx=0).stop 2024-01-20T08:31:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 -- 172.21.15.87:0/2748649637 shutdown_connections 2024-01-20T08:31:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 --2- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f0015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 --2- 172.21.15.87:0/2748649637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f00153af0 0x7f1f000c98d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 -- 172.21.15.87:0/2748649637 >> 172.21.15.87:0/2748649637 conn(0x7f1f0008b620 msgr2=0x7f1f00071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 -- 172.21.15.87:0/2748649637 shutdown_connections 2024-01-20T08:31:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.099+0000 7f1f0565d640 1 -- 172.21.15.87:0/2748649637 wait complete. 2024-01-20T08:31:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 Processor -- start 2024-01-20T08:31:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 -- start start 2024-01-20T08:31:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f0006c390 0x7f1f0015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f00156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f00153af0 0x7f1f00156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001ef060 con 0x7f1f00153af0 2024-01-20T08:31:28.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001ef250 con 0x7f1f0006c390 2024-01-20T08:31:28.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.101+0000 7f1f0565d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001ef530 con 0x7f1f0006c7b0 2024-01-20T08:31:28.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.102+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f00153af0 0x7f1f00156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.102+0000 7f1efe7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f00156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.102+0000 7f1efeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f0006c390 0x7f1f0015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.102+0000 7f1efe7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f00156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56766/0 (socket says 172.21.15.87:56766) 2024-01-20T08:31:28.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.102+0000 7f1efe7fc640 1 -- 172.21.15.87:0/1072415237 learned_addr learned my addr 172.21.15.87:0/1072415237 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.106+0000 7f1efeffd640 1 -- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 msgr2=0x7f1f00156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.106+0000 7f1efeffd640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f00156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.106+0000 7f1efeffd640 1 -- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f00153af0 msgr2=0x7f1f00156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.106+0000 7f1efeffd640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f00153af0 0x7f1f00156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.106+0000 7f1efeffd640 1 -- 172.21.15.87:0/1072415237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0009400 con 0x7f1f0006c390 2024-01-20T08:31:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.106+0000 7f1efe7fc640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f00156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:28.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.108+0000 7f1efeffd640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f0006c390 0x7f1f0015cf20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0033000 tx=0x7f1ef00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:28.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.111+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef003e070 con 0x7f1f0006c390 2024-01-20T08:31:28.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.111+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0006e10 con 0x7f1f0006c390 2024-01-20T08:31:28.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.111+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f001ef810 con 0x7f1f0006c390 2024-01-20T08:31:28.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.111+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0030ea0 con 0x7f1f0006c390 2024-01-20T08:31:28.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.111+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f001efdd0 con 0x7f1f0006c390 2024-01-20T08:31:28.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.112+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f001543a0 con 0x7f1f0006c390 2024-01-20T08:31:28.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.131+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1ef0003830 con 0x7f1f0006c390 2024-01-20T08:31:28.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.131+0000 7f1ee77fe640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1eec06af20 0x7f1eec06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.131+0000 7f1efe7fc640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1eec06af20 0x7f1eec06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.132+0000 7f1efe7fc640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1eec06af20 0x7f1eec06d3e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f00152d40 tx=0x7f1ef4009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:28.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.134+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ef007a4c0 con 0x7f1f0006c390 2024-01-20T08:31:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.365+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1ef0076800 con 0x7f1f0006c390 2024-01-20T08:31:28.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.551+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1f001543a0 con 0x7f1f0006c390 2024-01-20T08:31:28.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.565+0000 7f1ee77fe640 1 -- 172.21.15.87:0/1072415237 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1f001543a0 con 0x7f1f0006c390 2024-01-20T08:31:28.568 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1eec06af20 msgr2=0x7f1eec06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1eec06af20 0x7f1eec06d3e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f00152d40 tx=0x7f1ef4009040 comp rx=0 tx=0).stop 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f0006c390 msgr2=0x7f1f0015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f0006c390 0x7f1f0015cf20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0033000 tx=0x7f1ef00046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 shutdown_connections 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1eec06af20 0x7f1eec06d3e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f00153af0 0x7f1f00156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f0006c7b0 0x7f1f00156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 --2- 172.21.15.87:0/1072415237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f0006c390 0x7f1f0015cf20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 >> 172.21.15.87:0/1072415237 conn(0x7f1f0008b620 msgr2=0x7f1f00071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:28.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 shutdown_connections 2024-01-20T08:31:28.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.567+0000 7f1f0565d640 1 -- 172.21.15.87:0/1072415237 wait complete. 2024-01-20T08:31:28.581 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-20T08:31:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.679+0000 7f5afa3a9640 1 Processor -- start 2024-01-20T08:31:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.680+0000 7f5afa3a9640 1 -- start start 2024-01-20T08:31:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.680+0000 7f5afa3a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5af40c90e0 0x7f5af4157460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.680+0000 7f5afa3a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c9550 0x7f5af40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.680+0000 7f5afa3a9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40ca070 con 0x7f5af41517f0 2024-01-20T08:31:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.680+0000 7f5afa3a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40b0520 con 0x7f5af40c90e0 2024-01-20T08:31:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.680+0000 7f5afa3a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4151bd0 con 0x7f5af40c9550 2024-01-20T08:31:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.681+0000 7f5af37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5af40c90e0 0x7f5af4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.681+0000 7f5af8946640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c9550 0x7f5af40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.681+0000 7f5af3fff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5af41517f0 0x7f5af40b0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36794/0 (socket says 172.21.15.87:36794) 2024-01-20T08:31:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.681+0000 7f5af3fff640 1 -- 172.21.15.87:0/1057172202 learned_addr learned my addr 172.21.15.87:0/1057172202 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:28.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.684+0000 7f5af8946640 1 -- 172.21.15.87:0/1057172202 >> v2:172.21.15.87:3300/0 conn(0x7f5af40c90e0 msgr2=0x7f5af4157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:28.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.684+0000 7f5af8946640 1 --2- 172.21.15.87:0/1057172202 >> v2:172.21.15.87:3300/0 conn(0x7f5af40c90e0 0x7f5af4157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.684+0000 7f5af8946640 1 -- 172.21.15.87:0/1057172202 >> v1:172.21.15.87:6789/0 conn(0x7f5af41517f0 legacy=0x7f5af40b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.684+0000 7f5af8946640 1 -- 172.21.15.87:0/1057172202 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4157ef0 con 0x7f5af40c9550 2024-01-20T08:31:28.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.686+0000 7f5af8946640 1 --2- 172.21.15.87:0/1057172202 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c9550 0x7f5af40ccd00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009d90 tx=0x7f5aec02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133ffac4cdd1a158 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:28.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.690+0000 7f5af2ffd640 1 -- 172.21.15.87:0/1057172202 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec02cc50 con 0x7f5af40c9550 2024-01-20T08:31:28.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.690+0000 7f5af2ffd640 1 -- 172.21.15.87:0/1057172202 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec02cdb0 con 0x7f5af40c9550 2024-01-20T08:31:28.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.690+0000 7f5af2ffd640 1 -- 172.21.15.87:0/1057172202 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec034660 con 0x7f5af40c9550 2024-01-20T08:31:28.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1057172202 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c9550 msgr2=0x7f5af40ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1057172202 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c9550 0x7f5af40ccd00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009d90 tx=0x7f5aec02bea0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1057172202 shutdown_connections 2024-01-20T08:31:28.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1057172202 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c9550 0x7f5af40ccd00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1057172202 >> v2:172.21.15.87:3300/0 conn(0x7f5af40c90e0 0x7f5af4157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1057172202 >> 172.21.15.87:0/1057172202 conn(0x7f5af406a5b0 msgr2=0x7f5af406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:28.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1057172202 shutdown_connections 2024-01-20T08:31:28.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.691+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1057172202 wait complete. 2024-01-20T08:31:28.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.692+0000 7f5afa3a9640 1 Processor -- start 2024-01-20T08:31:28.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.693+0000 7f5afa3a9640 1 -- start start 2024-01-20T08:31:28.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5afa3a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 0x7f5af406cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5afa3a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5af40c9550 0x7f5af41f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5afa3a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af41517f0 0x7f5af40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5afa3a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af406c300 con 0x7f5af41517f0 2024-01-20T08:31:28.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5afa3a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af406b760 con 0x7f5af40c9550 2024-01-20T08:31:28.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5afa3a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40ad7b0 con 0x7f5af40c90e0 2024-01-20T08:31:28.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5af3fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 0x7f5af406cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5af3fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 0x7f5af406cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56786/0 (socket says 172.21.15.87:56786) 2024-01-20T08:31:28.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5af3fff640 1 -- 172.21.15.87:0/1662605084 learned_addr learned my addr 172.21.15.87:0/1662605084 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:28.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5af8946640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af41517f0 0x7f5af40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.694+0000 7f5af37fe640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5af40c9550 0x7f5af41f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.697+0000 7f5af3fff640 1 -- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5af40c9550 msgr2=0x7f5af41f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.697+0000 7f5af3fff640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5af40c9550 0x7f5af41f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.697+0000 7f5af3fff640 1 -- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af41517f0 msgr2=0x7f5af40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:28.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.697+0000 7f5af3fff640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af41517f0 0x7f5af40ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:28.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.698+0000 7f5af3fff640 1 -- 172.21.15.87:0/1662605084 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009a70 con 0x7f5af40c90e0 2024-01-20T08:31:28.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.702+0000 7f5af3fff640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 0x7f5af406cdb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800c400 tx=0x7f5ae800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:28.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.705+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8005ce0 con 0x7f5af40c90e0 2024-01-20T08:31:28.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.706+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8005e40 con 0x7f5af40c90e0 2024-01-20T08:31:28.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.706+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af40adb70 con 0x7f5af40c90e0 2024-01-20T08:31:28.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.706+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80072f0 con 0x7f5af40c90e0 2024-01-20T08:31:28.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.706+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af406bcb0 con 0x7f5af40c90e0 2024-01-20T08:31:28.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.707+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4074290 con 0x7f5af40c90e0 2024-01-20T08:31:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.949+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5ae8007510 con 0x7f5af40c90e0 2024-01-20T08:31:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.949+0000 7f5af0ff9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5ae006af20 0x7f5ae006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.949+0000 7f5af37fe640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5ae006af20 0x7f5ae006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.950+0000 7f5af37fe640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5ae006af20 0x7f5ae006d3e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5af41f20a0 tx=0x7f5ae4009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:28.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.954+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ae8057c80 con 0x7f5af40c90e0 2024-01-20T08:31:28.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:28.967+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5ae8053fc0 con 0x7f5af40c90e0 2024-01-20T08:31:29.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.147+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5af40c86b0 con 0x7f5af40c90e0 2024-01-20T08:31:29.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.151+0000 7f5af0ff9640 1 -- 172.21.15.87:0/1662605084 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ae8017070 con 0x7f5af40c90e0 2024-01-20T08:31:29.155 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5ae006af20 msgr2=0x7f5ae006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5ae006af20 0x7f5ae006d3e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5af41f20a0 tx=0x7f5ae4009190 comp rx=0 tx=0).stop 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 msgr2=0x7f5af406cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 0x7f5af406cdb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800c400 tx=0x7f5ae800c8c0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 shutdown_connections 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5ae006af20 0x7f5ae006d3e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af41517f0 0x7f5af40ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5af40c9550 0x7f5af41f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 --2- 172.21.15.87:0/1662605084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5af40c90e0 0x7f5af406cdb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.153+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 >> 172.21.15.87:0/1662605084 conn(0x7f5af406a5b0 msgr2=0x7f5af4070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.154+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 shutdown_connections 2024-01-20T08:31:29.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.154+0000 7f5afa3a9640 1 -- 172.21.15.87:0/1662605084 wait complete. 2024-01-20T08:31:29.168 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:31:29.168 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-20T08:31:29.183 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:29.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.320+0000 7f1ff170f640 1 Processor -- start 2024-01-20T08:31:29.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.320+0000 7f1ff170f640 1 -- start start 2024-01-20T08:31:29.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1ff170f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1ff170f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fec0ca090 0x7f1fec0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1ff170f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fec153eb0 con 0x7f1fec15d6f0 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1ff170f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fec15a560 con 0x7f1fec0ca090 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1ff170f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec15dc80 con 0x7f1fec153ad0 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1feb7fe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1fec15d6f0 0x7f1fec0c3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36810/0 (socket says 172.21.15.87:36810) 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1feb7fe640 1 -- 172.21.15.87:0/3620724089 learned_addr learned my addr 172.21.15.87:0/3620724089 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:29.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1feaffd640 1 --2- 172.21.15.87:0/3620724089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.321+0000 7f1fe3fff640 1 --2- 172.21.15.87:0/3620724089 >> v2:172.21.15.116:3300/0 conn(0x7f1fec0ca090 0x7f1fec0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.324+0000 7f1fea7fc640 1 -- 172.21.15.87:0/3620724089 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089771127 0 0) 0x7f1fe4001490 con 0x7f1fec15d6f0 2024-01-20T08:31:29.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.324+0000 7f1fea7fc640 1 -- 172.21.15.87:0/3620724089 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd8003520 con 0x7f1fec15d6f0 2024-01-20T08:31:29.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.325+0000 7f1feaffd640 1 -- 172.21.15.87:0/3620724089 >> v2:172.21.15.116:3300/0 conn(0x7f1fec0ca090 msgr2=0x7f1fec0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:29.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.325+0000 7f1feaffd640 1 --2- 172.21.15.87:0/3620724089 >> v2:172.21.15.116:3300/0 conn(0x7f1fec0ca090 0x7f1fec0ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.325+0000 7f1feaffd640 1 -- 172.21.15.87:0/3620724089 >> v1:172.21.15.87:6789/0 conn(0x7f1fec15d6f0 legacy=0x7f1fec0c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.325+0000 7f1feaffd640 1 -- 172.21.15.87:0/3620724089 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec0cb760 con 0x7f1fec153ad0 2024-01-20T08:31:29.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.327+0000 7f1feaffd640 1 --2- 172.21.15.87:0/3620724089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec0cb220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0097a0 tx=0x7f1fdc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f05076fea43ce07 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:29.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.331+0000 7f1fea7fc640 1 -- 172.21.15.87:0/3620724089 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc031830 con 0x7f1fec153ad0 2024-01-20T08:31:29.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.331+0000 7f1fea7fc640 1 -- 172.21.15.87:0/3620724089 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc031990 con 0x7f1fec153ad0 2024-01-20T08:31:29.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.331+0000 7f1fea7fc640 1 -- 172.21.15.87:0/3620724089 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc031d20 con 0x7f1fec153ad0 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 -- 172.21.15.87:0/3620724089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 msgr2=0x7f1fec0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 --2- 172.21.15.87:0/3620724089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec0cb220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0097a0 tx=0x7f1fdc02b8d0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 -- 172.21.15.87:0/3620724089 shutdown_connections 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 --2- 172.21.15.87:0/3620724089 >> v2:172.21.15.116:3300/0 conn(0x7f1fec0ca090 0x7f1fec0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 --2- 172.21.15.87:0/3620724089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec0cb220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 -- 172.21.15.87:0/3620724089 >> 172.21.15.87:0/3620724089 conn(0x7f1fec08af90 msgr2=0x7f1fec08b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 -- 172.21.15.87:0/3620724089 shutdown_connections 2024-01-20T08:31:29.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.332+0000 7f1ff170f640 1 -- 172.21.15.87:0/3620724089 wait complete. 2024-01-20T08:31:29.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.333+0000 7f1ff170f640 1 Processor -- start 2024-01-20T08:31:29.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.333+0000 7f1ff170f640 1 -- start start 2024-01-20T08:31:29.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1ff170f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec0ca090 0x7f1fec14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1ff170f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec1eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1ff170f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 0x7f1fec149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1ff170f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec15cab0 con 0x7f1fec15d6f0 2024-01-20T08:31:29.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1ff170f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec15cd00 con 0x7f1fec0ca090 2024-01-20T08:31:29.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1ff170f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec0cce30 con 0x7f1fec153ad0 2024-01-20T08:31:29.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1feb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 0x7f1fec149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1feaffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec0ca090 0x7f1fec14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.334+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec1eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.335+0000 7f1feb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 0x7f1fec149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49962/0 (socket says 172.21.15.87:49962) 2024-01-20T08:31:29.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.335+0000 7f1feb7fe640 1 -- 172.21.15.87:0/1286025861 learned_addr learned my addr 172.21.15.87:0/1286025861 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:29.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.338+0000 7f1feb7fe640 1 -- 172.21.15.87:0/1286025861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 msgr2=0x7f1fec1eeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.338+0000 7f1feb7fe640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec1eeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.338+0000 7f1feb7fe640 1 -- 172.21.15.87:0/1286025861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec0ca090 msgr2=0x7f1fec14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.338+0000 7f1feb7fe640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec0ca090 0x7f1fec14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.338+0000 7f1feb7fe640 1 -- 172.21.15.87:0/1286025861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc009400 con 0x7f1fec15d6f0 2024-01-20T08:31:29.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.340+0000 7f1feb7fe640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 0x7f1fec149870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400cae0 tx=0x7f1fe400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:29.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.344+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400ac40 con 0x7f1fec15d6f0 2024-01-20T08:31:29.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.344+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400ada0 con 0x7f1fec15d6f0 2024-01-20T08:31:29.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.344+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4012620 con 0x7f1fec15d6f0 2024-01-20T08:31:29.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.344+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec0cd110 con 0x7f1fec15d6f0 2024-01-20T08:31:29.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.344+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec0cddc0 con 0x7f1fec15d6f0 2024-01-20T08:31:29.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.345+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fec074290 con 0x7f1fec15d6f0 2024-01-20T08:31:29.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.363+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1fe4007070 con 0x7f1fec15d6f0 2024-01-20T08:31:29.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.364+0000 7f1fe37fe640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fcc06aed0 0x7f1fcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.364+0000 7f1feaffd640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fcc06aed0 0x7f1fcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.364+0000 7f1feaffd640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fcc06aed0 0x7f1fcc06d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc036000 tx=0x7f1fdc03c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:29.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.367+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1fe4058440 con 0x7f1fec15d6f0 2024-01-20T08:31:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.602+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1fe4054780 con 0x7f1fec15d6f0 2024-01-20T08:31:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.776+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fec153eb0 con 0x7f1fec15d6f0 2024-01-20T08:31:29.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.813+0000 7f1fe37fe640 1 -- 172.21.15.87:0/1286025861 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f1fec153eb0 con 0x7f1fec15d6f0 2024-01-20T08:31:29.817 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:29.817 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:29.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fcc06aed0 msgr2=0x7f1fcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fcc06aed0 0x7f1fcc06d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc036000 tx=0x7f1fdc03c040 comp rx=0 tx=0).stop 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 msgr2=0x7f1fec149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 0x7f1fec149870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400cae0 tx=0x7f1fe400a060 comp rx=0 tx=0).stop 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 shutdown_connections 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fcc06aed0 0x7f1fcc06d390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fec15d6f0 0x7f1fec149870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fec153ad0 0x7f1fec1eeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 --2- 172.21.15.87:0/1286025861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec0ca090 0x7f1fec14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 >> 172.21.15.87:0/1286025861 conn(0x7f1fec08af90 msgr2=0x7f1fec1511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 shutdown_connections 2024-01-20T08:31:29.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.815+0000 7f1ff170f640 1 -- 172.21.15.87:0/1286025861 wait complete. 2024-01-20T08:31:29.831 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-20T08:31:06.500533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-20T08:31:08.175669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-20T08:31:13.903374+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-20T08:31:29.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:31:29.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.930+0000 7efc1139e640 1 Processor -- start 2024-01-20T08:31:29.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.930+0000 7efc1139e640 1 -- start start 2024-01-20T08:31:29.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc1139e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 0x7efc0c0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc1139e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efc0c0ca8b0 0x7efc0c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc1139e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0c15a810 con 0x7efc0c0c9120 2024-01-20T08:31:29.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc1139e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0c0b4400 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc1139e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c0ca180 con 0x7efc0c15d940 2024-01-20T08:31:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc03fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 0x7efc0c0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.931+0000 7efc0b7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efc0c0ca8b0 0x7efc0c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.932+0000 7efc03fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 0x7efc0c0c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56814/0 (socket says 172.21.15.87:56814) 2024-01-20T08:31:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.932+0000 7efc03fff640 1 -- 172.21.15.87:0/3065439143 learned_addr learned my addr 172.21.15.87:0/3065439143 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:29.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.934+0000 7efc03fff640 1 -- 172.21.15.87:0/3065439143 >> v2:172.21.15.116:3300/0 conn(0x7efc0c0ca8b0 msgr2=0x7efc0c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:29.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.934+0000 7efc03fff640 1 --2- 172.21.15.87:0/3065439143 >> v2:172.21.15.116:3300/0 conn(0x7efc0c0ca8b0 0x7efc0c0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.934+0000 7efc03fff640 1 -- 172.21.15.87:0/3065439143 >> v1:172.21.15.116:6789/0 conn(0x7efc0c0c9120 legacy=0x7efc0c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.934+0000 7efc03fff640 1 -- 172.21.15.87:0/3065439143 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c154100 con 0x7efc0c15d940 2024-01-20T08:31:29.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.937+0000 7efc03fff640 1 --2- 172.21.15.87:0/3065439143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 0x7efc0c0c9c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009870 tx=0x7efbf402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616e427d15e6db51 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.941+0000 7efc0a7fc640 1 -- 172.21.15.87:0/3065439143 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4030840 con 0x7efc0c15d940 2024-01-20T08:31:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.941+0000 7efc0a7fc640 1 -- 172.21.15.87:0/3065439143 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf40309a0 con 0x7efc0c15d940 2024-01-20T08:31:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.941+0000 7efc0a7fc640 1 -- 172.21.15.87:0/3065439143 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4030d30 con 0x7efc0c15d940 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 -- 172.21.15.87:0/3065439143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 msgr2=0x7efc0c0c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 --2- 172.21.15.87:0/3065439143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 0x7efc0c0c9c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009870 tx=0x7efbf402b980 comp rx=0 tx=0).stop 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 -- 172.21.15.87:0/3065439143 shutdown_connections 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 --2- 172.21.15.87:0/3065439143 >> v2:172.21.15.116:3300/0 conn(0x7efc0c0ca8b0 0x7efc0c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 --2- 172.21.15.87:0/3065439143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c15d940 0x7efc0c0c9c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 -- 172.21.15.87:0/3065439143 >> 172.21.15.87:0/3065439143 conn(0x7efc0c08af90 msgr2=0x7efc0c08b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 -- 172.21.15.87:0/3065439143 shutdown_connections 2024-01-20T08:31:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.942+0000 7efc1139e640 1 -- 172.21.15.87:0/3065439143 wait complete. 2024-01-20T08:31:29.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.943+0000 7efc1139e640 1 Processor -- start 2024-01-20T08:31:29.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.943+0000 7efc1139e640 1 -- start start 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.943+0000 7efc1139e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c0c9120 0x7efc0c1f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc1139e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 0x7efc0c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc1139e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc0c15d940 0x7efc0c1f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc1139e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c1f3660 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc1139e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c0c28a0 con 0x7efc0c15d940 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc1139e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c1f0f40 con 0x7efc0c0c9120 2024-01-20T08:31:29.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc03fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 0x7efc0c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc0affd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c0c9120 0x7efc0c1f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.944+0000 7efc0b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc0c15d940 0x7efc0c1f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.945+0000 7efc03fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 0x7efc0c14e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49972/0 (socket says 172.21.15.87:49972) 2024-01-20T08:31:29.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.945+0000 7efc03fff640 1 -- 172.21.15.87:0/1631504143 learned_addr learned my addr 172.21.15.87:0/1631504143 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.948+0000 7efc03fff640 1 -- 172.21.15.87:0/1631504143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c0c9120 msgr2=0x7efc0c1f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.948+0000 7efc03fff640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c0c9120 0x7efc0c1f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.948+0000 7efc03fff640 1 -- 172.21.15.87:0/1631504143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc0c15d940 msgr2=0x7efc0c1f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.948+0000 7efc03fff640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc0c15d940 0x7efc0c1f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.948+0000 7efc03fff640 1 -- 172.21.15.87:0/1631504143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf40094d0 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.950+0000 7efc03fff640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 0x7efc0c14e370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:29.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.954+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4039c90 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.954+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf4039e10 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.954+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4043640 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.954+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c1f3380 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.954+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c1f0bd0 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.955+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0c1f0d90 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.973+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efbf40437a0 con 0x7efc0c0ca8b0 2024-01-20T08:31:29.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.973+0000 7efc037fe640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efbec06af20 0x7efbec06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:29.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.973+0000 7efc0affd640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efbec06af20 0x7efbec06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:29.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:29.974+0000 7efc0affd640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efbec06af20 0x7efbec06d3e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0016e0 tx=0x7efbfc009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:30.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.206+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efbf403e070 con 0x7efc0c0ca8b0 2024-01-20T08:31:30.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.206+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efbf4077ef0 con 0x7efc0c0ca8b0 2024-01-20T08:31:30.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.387+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc0c06d740 con 0x7efc0c0ca8b0 2024-01-20T08:31:30.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.392+0000 7efc037fe640 1 -- 172.21.15.87:0/1631504143 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7efbf4034070 con 0x7efc0c0ca8b0 2024-01-20T08:31:30.396 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efbec06af20 msgr2=0x7efbec06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efbec06af20 0x7efbec06d3e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0016e0 tx=0x7efbfc009040 comp rx=0 tx=0).stop 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 msgr2=0x7efc0c14e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 0x7efc0c14e370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf4004730 comp rx=0 tx=0).stop 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 shutdown_connections 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efbec06af20 0x7efbec06d3e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc0c15d940 0x7efc0c1f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc0c0ca8b0 0x7efc0c14e370 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 --2- 172.21.15.87:0/1631504143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efc0c0c9120 0x7efc0c1f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 >> 172.21.15.87:0/1631504143 conn(0x7efc0c08af90 msgr2=0x7efc0c0c70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 shutdown_connections 2024-01-20T08:31:30.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.394+0000 7efc1139e640 1 -- 172.21.15.87:0/1631504143 wait complete. 2024-01-20T08:31:30.409 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-20T08:31:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.507+0000 7f0d922ee640 1 Processor -- start 2024-01-20T08:31:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.507+0000 7f0d922ee640 1 -- start start 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.508+0000 7f0d922ee640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d8c153740 0x7f0d8c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.508+0000 7f0d922ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c0c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.508+0000 7f0d922ee640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c06e350 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.508+0000 7f0d922ee640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c06ddd0 con 0x7f0d8c153740 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.508+0000 7f0d922ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c06ffc0 con 0x7f0d8c0c88f0 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.509+0000 7f0d8bfff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d8c153740 0x7f0d8c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.509+0000 7f0d9088b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c0c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.509+0000 7f0d8b7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0d8c0c2d10 0x7f0d8c06eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:43348/0 (socket says 172.21.15.87:43348) 2024-01-20T08:31:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.509+0000 7f0d8b7fe640 1 -- 172.21.15.87:0/3189306446 learned_addr learned my addr 172.21.15.87:0/3189306446 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.511+0000 7f0d8bfff640 1 -- 172.21.15.87:0/3189306446 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 msgr2=0x7f0d8c0c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.511+0000 7f0d8bfff640 1 --2- 172.21.15.87:0/3189306446 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c0c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.511+0000 7f0d8bfff640 1 -- 172.21.15.87:0/3189306446 >> v1:172.21.15.116:6789/0 conn(0x7f0d8c0c2d10 legacy=0x7f0d8c06eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.511+0000 7f0d8bfff640 1 -- 172.21.15.87:0/3189306446 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c070280 con 0x7f0d8c153740 2024-01-20T08:31:30.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.513+0000 7f0d8bfff640 1 --2- 172.21.15.87:0/3189306446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c153740 0x7f0d8c0c71e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0097a0 tx=0x7f0d7c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81a6a4d3c0b2cf0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.517+0000 7f0d8affd640 1 -- 172.21.15.87:0/3189306446 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c030610 con 0x7f0d8c153740 2024-01-20T08:31:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.517+0000 7f0d8affd640 1 -- 172.21.15.87:0/3189306446 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c030770 con 0x7f0d8c153740 2024-01-20T08:31:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.517+0000 7f0d8affd640 1 -- 172.21.15.87:0/3189306446 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c030b00 con 0x7f0d8c153740 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 -- 172.21.15.87:0/3189306446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c153740 msgr2=0x7f0d8c0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 --2- 172.21.15.87:0/3189306446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c153740 0x7f0d8c0c71e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0097a0 tx=0x7f0d7c02b960 comp rx=0 tx=0).stop 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 -- 172.21.15.87:0/3189306446 shutdown_connections 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 --2- 172.21.15.87:0/3189306446 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c0c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 --2- 172.21.15.87:0/3189306446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c153740 0x7f0d8c0c71e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 -- 172.21.15.87:0/3189306446 >> 172.21.15.87:0/3189306446 conn(0x7f0d8c06b6c0 msgr2=0x7f0d8c06baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 -- 172.21.15.87:0/3189306446 shutdown_connections 2024-01-20T08:31:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.518+0000 7f0d922ee640 1 -- 172.21.15.87:0/3189306446 wait complete. 2024-01-20T08:31:30.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.519+0000 7f0d922ee640 1 Processor -- start 2024-01-20T08:31:30.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.519+0000 7f0d922ee640 1 -- start start 2024-01-20T08:31:30.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d922ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c0c2d10 0x7f0d8c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:30.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d922ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:30.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d922ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d8c153740 0x7f0d8c148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:30.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d922ee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c151920 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d922ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c1eed40 con 0x7f0d8c153740 2024-01-20T08:31:30.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d922ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c152f30 con 0x7f0d8c0c88f0 2024-01-20T08:31:30.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d9088b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d8c153740 0x7f0d8c148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:30.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c0c2d10 0x7f0d8c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:30.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.520+0000 7f0d8b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:30.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.521+0000 7f0d8b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c1ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56840/0 (socket says 172.21.15.87:56840) 2024-01-20T08:31:30.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.521+0000 7f0d8b7fe640 1 -- 172.21.15.87:0/1425250033 learned_addr learned my addr 172.21.15.87:0/1425250033 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:30.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.524+0000 7f0d8bfff640 1 -- 172.21.15.87:0/1425250033 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 msgr2=0x7f0d8c1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.524+0000 7f0d8bfff640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.524+0000 7f0d8bfff640 1 -- 172.21.15.87:0/1425250033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d8c153740 msgr2=0x7f0d8c148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.524+0000 7f0d8bfff640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d8c153740 0x7f0d8c148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.524+0000 7f0d8bfff640 1 -- 172.21.15.87:0/1425250033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009400 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.526+0000 7f0d8bfff640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c0c2d10 0x7f0d8c1586a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c033000 tx=0x7f0d7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:30.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.530+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c03e040 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.530+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c039cb0 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.530+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c044630 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.530+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c151080 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.530+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c151640 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.531+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c074290 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.550+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0d7c02f370 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.550+0000 7f0d88ff9640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d7806aed0 0x7f0d7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:30.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.550+0000 7f0d8b7fe640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d7806aed0 0x7f0d7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:30.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.550+0000 7f0d8b7fe640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d7806aed0 0x7f0d7806d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0cb4e0 tx=0x7f0d80009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:30.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.784+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0d7c039e20 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.784+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0d7c076c30 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.966+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d8c06ee00 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.971+0000 7f0d88ff9640 1 -- 172.21.15.87:0/1425250033 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d7c0340a0 con 0x7f0d8c0c2d10 2024-01-20T08:31:30.975 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d7806aed0 msgr2=0x7f0d7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d7806aed0 0x7f0d7806d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0cb4e0 tx=0x7f0d80009040 comp rx=0 tx=0).stop 2024-01-20T08:31:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c0c2d10 msgr2=0x7f0d8c1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c0c2d10 0x7f0d8c1586a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c033000 tx=0x7f0d7c004730 comp rx=0 tx=0).stop 2024-01-20T08:31:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 shutdown_connections 2024-01-20T08:31:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d7806aed0 0x7f0d7806d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d8c153740 0x7f0d8c148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d8c0c88f0 0x7f0d8c1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 --2- 172.21.15.87:0/1425250033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d8c0c2d10 0x7f0d8c1586a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 >> 172.21.15.87:0/1425250033 conn(0x7f0d8c06b6c0 msgr2=0x7f0d8c0cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 shutdown_connections 2024-01-20T08:31:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:30.973+0000 7f0d922ee640 1 -- 172.21.15.87:0/1425250033 wait complete. 2024-01-20T08:31:30.988 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-20T08:31:31.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.086+0000 7f3232966640 1 Processor -- start 2024-01-20T08:31:31.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.086+0000 7f3232966640 1 -- start start 2024-01-20T08:31:31.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.087+0000 7f3232966640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c1517f0 0x7f322c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.087+0000 7f3232966640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0ca070 con 0x7f322c0c90e0 2024-01-20T08:31:31.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.087+0000 7f3232966640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c157ef0 con 0x7f322c0c9520 2024-01-20T08:31:31.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.087+0000 7f3232966640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b0520 con 0x7f322c1517f0 2024-01-20T08:31:31.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.088+0000 7f322bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c1517f0 0x7f322c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.088+0000 7f322b7fe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f322c0c90e0 0x7f322c157460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36816/0 (socket says 172.21.15.87:36816) 2024-01-20T08:31:31.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.088+0000 7f322b7fe640 1 -- 172.21.15.87:0/2795483199 learned_addr learned my addr 172.21.15.87:0/2795483199 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:31.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.090+0000 7f322affd640 1 -- 172.21.15.87:0/2795483199 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227101054 0 0) 0x7f322c0ca070 con 0x7f322c0c90e0 2024-01-20T08:31:31.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.090+0000 7f322affd640 1 -- 172.21.15.87:0/2795483199 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3214003520 con 0x7f322c0c90e0 2024-01-20T08:31:31.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.090+0000 7f322bfff640 1 -- 172.21.15.87:0/2795483199 >> v1:172.21.15.116:6789/0 conn(0x7f322c0c9520 legacy=0x7f322c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.090+0000 7f322bfff640 1 -- 172.21.15.87:0/2795483199 >> v1:172.21.15.87:6789/0 conn(0x7f322c0c90e0 legacy=0x7f322c157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.090+0000 7f322bfff640 1 -- 172.21.15.87:0/2795483199 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c1581d0 con 0x7f322c1517f0 2024-01-20T08:31:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.093+0000 7f322bfff640 1 --2- 172.21.15.87:0/2795483199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c1517f0 0x7f322c0b0dd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f321c0097d0 tx=0x7f321c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=112a6cb92be66bbc server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:31.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.096+0000 7f322affd640 1 -- 172.21.15.87:0/2795483199 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c031890 con 0x7f322c1517f0 2024-01-20T08:31:31.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.096+0000 7f322affd640 1 -- 172.21.15.87:0/2795483199 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0319f0 con 0x7f322c1517f0 2024-01-20T08:31:31.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.096+0000 7f322affd640 1 -- 172.21.15.87:0/2795483199 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c031d80 con 0x7f322c1517f0 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 -- 172.21.15.87:0/2795483199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c1517f0 msgr2=0x7f322c0b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 --2- 172.21.15.87:0/2795483199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c1517f0 0x7f322c0b0dd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f321c0097d0 tx=0x7f321c02b930 comp rx=0 tx=0).stop 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 -- 172.21.15.87:0/2795483199 shutdown_connections 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 --2- 172.21.15.87:0/2795483199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c1517f0 0x7f322c0b0dd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 -- 172.21.15.87:0/2795483199 >> 172.21.15.87:0/2795483199 conn(0x7f322c06a5b0 msgr2=0x7f322c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 -- 172.21.15.87:0/2795483199 shutdown_connections 2024-01-20T08:31:31.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.097+0000 7f3232966640 1 -- 172.21.15.87:0/2795483199 wait complete. 2024-01-20T08:31:31.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.098+0000 7f3232966640 1 Processor -- start 2024-01-20T08:31:31.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.098+0000 7f3232966640 1 -- start start 2024-01-20T08:31:31.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3232966640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c0c90e0 0x7f322c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3232966640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f322c0c9520 0x7f322c0aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3232966640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 0x7f322c0af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3232966640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c14c890 con 0x7f322c1517f0 2024-01-20T08:31:31.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3232966640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0ae790 con 0x7f322c0c9520 2024-01-20T08:31:31.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3232966640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0adbf0 con 0x7f322c0c90e0 2024-01-20T08:31:31.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f3230f03640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 0x7f322c0af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f322c0c9520 0x7f322c0aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.099+0000 7f322bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c0c90e0 0x7f322c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.100+0000 7f322bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c0c90e0 0x7f322c14bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56856/0 (socket says 172.21.15.87:56856) 2024-01-20T08:31:31.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.100+0000 7f3230f03640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 0x7f322c0af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49994/0 (socket says 172.21.15.87:49994) 2024-01-20T08:31:31.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.100+0000 7f322bfff640 1 -- 172.21.15.87:0/1535759977 learned_addr learned my addr 172.21.15.87:0/1535759977 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:31.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.103+0000 7f3230f03640 1 -- 172.21.15.87:0/1535759977 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c0c90e0 msgr2=0x7f322c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.103+0000 7f3230f03640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c0c90e0 0x7f322c14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.103+0000 7f3230f03640 1 -- 172.21.15.87:0/1535759977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f322c0c9520 msgr2=0x7f322c0aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.103+0000 7f3230f03640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f322c0c9520 0x7f322c0aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.103+0000 7f3230f03640 1 -- 172.21.15.87:0/1535759977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c009400 con 0x7f322c1517f0 2024-01-20T08:31:31.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.105+0000 7f3230f03640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 0x7f322c0af160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f322400c690 tx=0x7f322400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.109+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224003b70 con 0x7f322c1517f0 2024-01-20T08:31:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.109+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3224003cd0 con 0x7f322c1517f0 2024-01-20T08:31:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.109+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224005310 con 0x7f322c1517f0 2024-01-20T08:31:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.109+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c06b6c0 con 0x7f322c1517f0 2024-01-20T08:31:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.109+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0ae140 con 0x7f322c1517f0 2024-01-20T08:31:31.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.110+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c074290 con 0x7f322c1517f0 2024-01-20T08:31:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.128+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3224005470 con 0x7f322c1517f0 2024-01-20T08:31:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.128+0000 7f3228ff9640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321806af20 0x7f321806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.129+0000 7f322bfff640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321806af20 0x7f321806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.129+0000 7f322bfff640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321806af20 0x7f321806d3e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c03c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:31.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.132+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f32240588d0 con 0x7f322c1517f0 2024-01-20T08:31:31.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.363+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3224054c10 con 0x7f322c1517f0 2024-01-20T08:31:31.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.531+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f322c0ae8d0 con 0x7f322c1517f0 2024-01-20T08:31:31.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.537+0000 7f3228ff9640 1 -- 172.21.15.87:0/1535759977 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3224058230 con 0x7f322c1517f0 2024-01-20T08:31:31.541 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321806af20 msgr2=0x7f321806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321806af20 0x7f321806d3e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c03c040 comp rx=0 tx=0).stop 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 msgr2=0x7f322c0af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 0x7f322c0af160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f322400c690 tx=0x7f322400cb50 comp rx=0 tx=0).stop 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 shutdown_connections 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321806af20 0x7f321806d3e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c1517f0 0x7f322c0af160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f322c0c9520 0x7f322c0aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 --2- 172.21.15.87:0/1535759977 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f322c0c90e0 0x7f322c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 >> 172.21.15.87:0/1535759977 conn(0x7f322c06a5b0 msgr2=0x7f322c14fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:31.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 shutdown_connections 2024-01-20T08:31:31.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.539+0000 7f3232966640 1 -- 172.21.15.87:0/1535759977 wait complete. 2024-01-20T08:31:31.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-20T08:31:31.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.653+0000 7f6081258640 1 Processor -- start 2024-01-20T08:31:31.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.653+0000 7f6081258640 1 -- start start 2024-01-20T08:31:31.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f6081258640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c0c4b10 0x7f607c071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f6081258640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c157e40 con 0x7f607c151690 2024-01-20T08:31:31.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f6081258640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c06da50 con 0x7f607c151a70 2024-01-20T08:31:31.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f6081258640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0c73d0 con 0x7f607c0c4b10 2024-01-20T08:31:31.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f607ad9d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f607c151690 0x7f607c06e6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36820/0 (socket says 172.21.15.87:36820) 2024-01-20T08:31:31.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f607ad9d640 1 -- 172.21.15.87:0/216504774 learned_addr learned my addr 172.21.15.87:0/216504774 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:31.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.654+0000 7f607b59e640 1 --2- 172.21.15.87:0/216504774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c0c4b10 0x7f607c071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634164 0 0) 0x7f607c157e40 con 0x7f607c151690 2024-01-20T08:31:31.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6068003520 con 0x7f607c151690 2024-01-20T08:31:31.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810299875 0 0) 0x7f607c06da50 con 0x7f607c151a70 2024-01-20T08:31:31.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c157e40 con 0x7f607c151a70 2024-01-20T08:31:31.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607b59e640 1 -- 172.21.15.87:0/216504774 >> v1:172.21.15.116:6789/0 conn(0x7f607c151a70 legacy=0x7f607c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607b59e640 1 -- 172.21.15.87:0/216504774 >> v1:172.21.15.87:6789/0 conn(0x7f607c151690 legacy=0x7f607c06e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.657+0000 7f607b59e640 1 -- 172.21.15.87:0/216504774 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c157530 con 0x7f607c0c4b10 2024-01-20T08:31:31.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.660+0000 7f607b59e640 1 --2- 172.21.15.87:0/216504774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c0c4b10 0x7f607c071c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6074009dc0 tx=0x7f607402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e9bbb0589d57d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:31.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607402cd50 con 0x7f607c0c4b10 2024-01-20T08:31:31.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6074034440 con 0x7f607c0c4b10 2024-01-20T08:31:31.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f607a59c640 1 -- 172.21.15.87:0/216504774 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60740347d0 con 0x7f607c0c4b10 2024-01-20T08:31:31.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 -- 172.21.15.87:0/216504774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c0c4b10 msgr2=0x7f607c071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 --2- 172.21.15.87:0/216504774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c0c4b10 0x7f607c071c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6074009dc0 tx=0x7f607402bfa0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 -- 172.21.15.87:0/216504774 shutdown_connections 2024-01-20T08:31:31.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 --2- 172.21.15.87:0/216504774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c0c4b10 0x7f607c071c50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 -- 172.21.15.87:0/216504774 >> 172.21.15.87:0/216504774 conn(0x7f607c06a9a0 msgr2=0x7f607c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:31.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 -- 172.21.15.87:0/216504774 shutdown_connections 2024-01-20T08:31:31.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.664+0000 7f6081258640 1 -- 172.21.15.87:0/216504774 wait complete. 2024-01-20T08:31:31.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 Processor -- start 2024-01-20T08:31:31.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 -- start start 2024-01-20T08:31:31.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 0x7f607c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f607c151690 0x7f607c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c151a70 0x7f607c1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0c9990 con 0x7f607c0c4b10 2024-01-20T08:31:31.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c144500 con 0x7f607c151690 2024-01-20T08:31:31.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.666+0000 7f6081258640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c1edb40 con 0x7f607c151a70 2024-01-20T08:31:31.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.667+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 0x7f607c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.667+0000 7f607b59e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c151a70 0x7f607c1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.667+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 0x7f607c154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50004/0 (socket says 172.21.15.87:50004) 2024-01-20T08:31:31.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.667+0000 7f607ad9d640 1 -- 172.21.15.87:0/1630858212 learned_addr learned my addr 172.21.15.87:0/1630858212 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:31.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.667+0000 7f6073fff640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f607c151690 0x7f607c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.671+0000 7f607ad9d640 1 -- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c151a70 msgr2=0x7f607c1521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.671+0000 7f607ad9d640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c151a70 0x7f607c1521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.671+0000 7f607ad9d640 1 -- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f607c151690 msgr2=0x7f607c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:31.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.671+0000 7f607ad9d640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f607c151690 0x7f607c150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:31.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.671+0000 7f607ad9d640 1 -- 172.21.15.87:0/1630858212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074009a70 con 0x7f607c0c4b10 2024-01-20T08:31:31.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.673+0000 7f607ad9d640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 0x7f607c154050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c500 tx=0x7f606c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.677+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c005ce0 con 0x7f607c0c4b10 2024-01-20T08:31:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.677+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c005e60 con 0x7f607c0c4b10 2024-01-20T08:31:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.677+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c007330 con 0x7f607c0c4b10 2024-01-20T08:31:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.677+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c0c8d10 con 0x7f607c0c4b10 2024-01-20T08:31:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.677+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c0c92d0 con 0x7f607c0c4b10 2024-01-20T08:31:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.678+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c0c9ad0 con 0x7f607c0c4b10 2024-01-20T08:31:31.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.922+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f606c007490 con 0x7f607c0c4b10 2024-01-20T08:31:31.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.922+0000 7f60737fe640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f605c06af20 0x7f605c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:31.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.922+0000 7f6073fff640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f605c06af20 0x7f605c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:31.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.923+0000 7f6073fff640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f605c06af20 0x7f605c06d3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f607c156230 tx=0x7f6064009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:31.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.936+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f606c057c30 con 0x7f607c0c4b10 2024-01-20T08:31:31.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:31.936+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f606c053f70 con 0x7f607c0c4b10 2024-01-20T08:31:32.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.116+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f607c074290 con 0x7f607c0c4b10 2024-01-20T08:31:32.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.121+0000 7f60737fe640 1 -- 172.21.15.87:0/1630858212 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f606c017070 con 0x7f607c0c4b10 2024-01-20T08:31:32.125 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f605c06af20 msgr2=0x7f605c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f605c06af20 0x7f605c06d3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f607c156230 tx=0x7f6064009040 comp rx=0 tx=0).stop 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 msgr2=0x7f607c154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 0x7f607c154050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c500 tx=0x7f606c00c9c0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 shutdown_connections 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f605c06af20 0x7f605c06d3e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f607c151a70 0x7f607c1521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f607c151690 0x7f607c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 --2- 172.21.15.87:0/1630858212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0c4b10 0x7f607c154050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 >> 172.21.15.87:0/1630858212 conn(0x7f607c06a9a0 msgr2=0x7f607c0c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:32.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 shutdown_connections 2024-01-20T08:31:32.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.123+0000 7f6081258640 1 -- 172.21.15.87:0/1630858212 wait complete. 2024-01-20T08:31:32.138 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-20T08:31:32.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.236+0000 7f48ea31d640 1 Processor -- start 2024-01-20T08:31:32.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.236+0000 7f48ea31d640 1 -- start start 2024-01-20T08:31:32.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.237+0000 7f48ea31d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48e4151690 0x7f48e4071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.237+0000 7f48ea31d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48e4151a70 0x7f48e406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.237+0000 7f48ea31d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e4157e40 con 0x7f48e40c4b10 2024-01-20T08:31:32.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.237+0000 7f48ea31d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40c73d0 con 0x7f48e4151a70 2024-01-20T08:31:32.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.237+0000 7f48ea31d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e406da50 con 0x7f48e4151690 2024-01-20T08:31:32.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.238+0000 7f48e3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48e4151690 0x7f48e4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.238+0000 7f48e37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48e4151a70 0x7f48e406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.238+0000 7f48e37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48e4151a70 0x7f48e406e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50006/0 (socket says 172.21.15.87:50006) 2024-01-20T08:31:32.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.238+0000 7f48e37fe640 1 -- 172.21.15.87:0/2850146592 learned_addr learned my addr 172.21.15.87:0/2850146592 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:32.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.240+0000 7f48e37fe640 1 -- 172.21.15.87:0/2850146592 >> v2:172.21.15.116:3300/0 conn(0x7f48e4151690 msgr2=0x7f48e4071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:32.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.240+0000 7f48e37fe640 1 --2- 172.21.15.87:0/2850146592 >> v2:172.21.15.116:3300/0 conn(0x7f48e4151690 0x7f48e4071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.240+0000 7f48e37fe640 1 -- 172.21.15.87:0/2850146592 >> v1:172.21.15.116:6789/0 conn(0x7f48e40c4b10 legacy=0x7f48e40c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.240+0000 7f48e37fe640 1 -- 172.21.15.87:0/2850146592 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4157530 con 0x7f48e4151a70 2024-01-20T08:31:32.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.242+0000 7f48e37fe640 1 --2- 172.21.15.87:0/2850146592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e406e6a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f48e406d770 tx=0x7f48d802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7124a68c75d3c66a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:32.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48e2ffd640 1 -- 172.21.15.87:0/2850146592 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d802f850 con 0x7f48e4151a70 2024-01-20T08:31:32.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48e2ffd640 1 -- 172.21.15.87:0/2850146592 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d802f9b0 con 0x7f48e4151a70 2024-01-20T08:31:32.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48e2ffd640 1 -- 172.21.15.87:0/2850146592 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d802fd40 con 0x7f48e4151a70 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 -- 172.21.15.87:0/2850146592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 msgr2=0x7f48e406e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 --2- 172.21.15.87:0/2850146592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e406e6a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f48e406d770 tx=0x7f48d802ba30 comp rx=0 tx=0).stop 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 -- 172.21.15.87:0/2850146592 shutdown_connections 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 --2- 172.21.15.87:0/2850146592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e406e6a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 --2- 172.21.15.87:0/2850146592 >> v2:172.21.15.116:3300/0 conn(0x7f48e4151690 0x7f48e4071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 -- 172.21.15.87:0/2850146592 >> 172.21.15.87:0/2850146592 conn(0x7f48e406a9a0 msgr2=0x7f48e406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.246+0000 7f48ea31d640 1 -- 172.21.15.87:0/2850146592 shutdown_connections 2024-01-20T08:31:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.247+0000 7f48ea31d640 1 -- 172.21.15.87:0/2850146592 wait complete. 2024-01-20T08:31:32.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 Processor -- start 2024-01-20T08:31:32.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 -- start start 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e40c4b10 0x7f48e41f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48e4151690 0x7f48e40ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e414ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4154920 con 0x7f48e4151a70 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e414bca0 con 0x7f48e40c4b10 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.248+0000 7f48ea31d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41f0f50 con 0x7f48e4151690 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.249+0000 7f48e88ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e414ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.249+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e40c4b10 0x7f48e41f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.249+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48e4151690 0x7f48e40ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.249+0000 7f48e88ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e414ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50018/0 (socket says 172.21.15.87:50018) 2024-01-20T08:31:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.249+0000 7f48e88ba640 1 -- 172.21.15.87:0/3092430721 learned_addr learned my addr 172.21.15.87:0/3092430721 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.253+0000 7f48e3fff640 1 -- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48e4151690 msgr2=0x7f48e40ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.253+0000 7f48e3fff640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48e4151690 0x7f48e40ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.253+0000 7f48e3fff640 1 -- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 msgr2=0x7f48e414ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.253+0000 7f48e3fff640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e414ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.253+0000 7f48e3fff640 1 -- 172.21.15.87:0/3092430721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d80094d0 con 0x7f48e40c4b10 2024-01-20T08:31:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.253+0000 7f48e88ba640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e414ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:32.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.255+0000 7f48e3fff640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e40c4b10 0x7f48e41f0a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e880 tx=0x7f48d400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:32.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.259+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400c8e0 con 0x7f48e40c4b10 2024-01-20T08:31:32.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.259+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400ca40 con 0x7f48e40c4b10 2024-01-20T08:31:32.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.259+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400cdf0 con 0x7f48e40c4b10 2024-01-20T08:31:32.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.259+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e41edf00 con 0x7f48e40c4b10 2024-01-20T08:31:32.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.259+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41ee7a0 con 0x7f48e40c4b10 2024-01-20T08:31:32.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.260+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4154e90 con 0x7f48e40c4b10 2024-01-20T08:31:32.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.279+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f48d4013470 con 0x7f48e40c4b10 2024-01-20T08:31:32.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.279+0000 7f48e0ff9640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f48d006af20 0x7f48d006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.279+0000 7f48e37fe640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f48d006af20 0x7f48d006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.280+0000 7f48e37fe640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f48d006af20 0x7f48d006d3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48d8036000 tx=0x7f48d8008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:32.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.281+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f48d4058b30 con 0x7f48e40c4b10 2024-01-20T08:31:32.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.514+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f48d4054e70 con 0x7f48e40c4b10 2024-01-20T08:31:32.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.700+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f48e4074290 con 0x7f48e40c4b10 2024-01-20T08:31:32.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.706+0000 7f48e0ff9640 1 -- 172.21.15.87:0/3092430721 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f48d4021020 con 0x7f48e40c4b10 2024-01-20T08:31:32.710 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-01-20T08:31:32.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f48d006af20 msgr2=0x7f48d006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f48d006af20 0x7f48d006d3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48d8036000 tx=0x7f48d8008030 comp rx=0 tx=0).stop 2024-01-20T08:31:32.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e40c4b10 msgr2=0x7f48e41f0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e40c4b10 0x7f48e41f0a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e880 tx=0x7f48d400ed40 comp rx=0 tx=0).stop 2024-01-20T08:31:32.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 shutdown_connections 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f48d006af20 0x7f48d006d3e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151a70 0x7f48e414ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48e4151690 0x7f48e40ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 --2- 172.21.15.87:0/3092430721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e40c4b10 0x7f48e41f0a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 >> 172.21.15.87:0/3092430721 conn(0x7f48e406a9a0 msgr2=0x7f48e41569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 shutdown_connections 2024-01-20T08:31:32.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.708+0000 7f48ea31d640 1 -- 172.21.15.87:0/3092430721 wait complete. 2024-01-20T08:31:32.724 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-01-20T08:31:32.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:32.724 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-01-20T08:31:32.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:ceph-fuse[43157]: fuse finished with error 0 and tester_r 0 2024-01-20T08:31:32.751 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T08:31:32.794 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:31:32.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:31:32.795 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-01-20T08:31:32.823 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:32.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.960+0000 7f1fcc677640 1 Processor -- start 2024-01-20T08:31:32.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.961+0000 7f1fcc677640 1 -- start start 2024-01-20T08:31:32.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.961+0000 7f1fcc677640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1fc414fde0 0x7f1fc40727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.961+0000 7f1fcc677640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fc41590e0 0x7f1fc406e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.961+0000 7f1fcc677640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc4156070 con 0x7f1fc40721b0 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.961+0000 7f1fcc677640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc4155af0 con 0x7f1fc414fde0 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.961+0000 7f1fcc677640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc406eb90 con 0x7f1fc41590e0 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.962+0000 7f1fcac14640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fc41590e0 0x7f1fc406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.962+0000 7f1fcac14640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fc41590e0 0x7f1fc406e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:60944/0 (socket says 172.21.15.87:60944) 2024-01-20T08:31:32.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.962+0000 7f1fcac14640 1 -- 172.21.15.87:0/2246929911 learned_addr learned my addr 172.21.15.87:0/2246929911 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:32.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.965+0000 7f1fcac14640 1 -- 172.21.15.87:0/2246929911 >> v2:172.21.15.87:3300/0 conn(0x7f1fc414fde0 msgr2=0x7f1fc40727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:32.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.965+0000 7f1fcac14640 1 --2- 172.21.15.87:0/2246929911 >> v2:172.21.15.87:3300/0 conn(0x7f1fc414fde0 0x7f1fc40727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.965+0000 7f1fcac14640 1 -- 172.21.15.87:0/2246929911 >> v1:172.21.15.87:6789/0 conn(0x7f1fc40721b0 legacy=0x7f1fc4156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:32.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.965+0000 7f1fcac14640 1 -- 172.21.15.87:0/2246929911 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc41501c0 con 0x7f1fc41590e0 2024-01-20T08:31:32.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.967+0000 7f1fcac14640 1 --2- 172.21.15.87:0/2246929911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc41590e0 0x7f1fc406e650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40c5a00 tx=0x7f1fbc02bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99723731cd77783 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:32.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.970+0000 7f1fc9411640 1 -- 172.21.15.87:0/2246929911 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc02cb20 con 0x7f1fc41590e0 2024-01-20T08:31:32.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.970+0000 7f1fc9411640 1 -- 172.21.15.87:0/2246929911 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc02cc80 con 0x7f1fc41590e0 2024-01-20T08:31:32.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.970+0000 7f1fc9411640 1 -- 172.21.15.87:0/2246929911 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0343f0 con 0x7f1fc41590e0 2024-01-20T08:31:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 -- 172.21.15.87:0/2246929911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc41590e0 msgr2=0x7f1fc406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 --2- 172.21.15.87:0/2246929911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc41590e0 0x7f1fc406e650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40c5a00 tx=0x7f1fbc02bf80 comp rx=0 tx=0).stop 2024-01-20T08:31:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 -- 172.21.15.87:0/2246929911 shutdown_connections 2024-01-20T08:31:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 --2- 172.21.15.87:0/2246929911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc41590e0 0x7f1fc406e650 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 --2- 172.21.15.87:0/2246929911 >> v2:172.21.15.87:3300/0 conn(0x7f1fc414fde0 0x7f1fc40727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 -- 172.21.15.87:0/2246929911 >> 172.21.15.87:0/2246929911 conn(0x7f1fc406d510 msgr2=0x7f1fc40663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 -- 172.21.15.87:0/2246929911 shutdown_connections 2024-01-20T08:31:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.971+0000 7f1fcc677640 1 -- 172.21.15.87:0/2246929911 wait complete. 2024-01-20T08:31:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.972+0000 7f1fcc677640 1 Processor -- start 2024-01-20T08:31:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.972+0000 7f1fcc677640 1 -- start start 2024-01-20T08:31:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fcc677640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc40721b0 0x7f1fc41505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fcc677640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 0x7f1fc4158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fcc677640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc41590e0 0x7f1fc40c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fcc677640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41ed580 con 0x7f1fc40721b0 2024-01-20T08:31:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fcc677640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40c3000 con 0x7f1fc414fde0 2024-01-20T08:31:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fcc677640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40c32e0 con 0x7f1fc41590e0 2024-01-20T08:31:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.973+0000 7f1fc9c12640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 0x7f1fc4158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.974+0000 7f1fcac14640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc41590e0 0x7f1fc40c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.974+0000 7f1fc9c12640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 0x7f1fc4158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:60950/0 (socket says 172.21.15.87:60950) 2024-01-20T08:31:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.974+0000 7f1fc9c12640 1 -- 172.21.15.87:0/35932781 learned_addr learned my addr 172.21.15.87:0/35932781 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.977+0000 7f1fc9c12640 1 -- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc41590e0 msgr2=0x7f1fc40c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.977+0000 7f1fc9c12640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc41590e0 0x7f1fc40c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.977+0000 7f1fc9c12640 1 -- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc40721b0 msgr2=0x7f1fc41505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.977+0000 7f1fc9c12640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc40721b0 0x7f1fc41505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.977+0000 7f1fc9c12640 1 -- 172.21.15.87:0/35932781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009b30 con 0x7f1fc414fde0 2024-01-20T08:31:32.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.979+0000 7f1fc9c12640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 0x7f1fc4158180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c3f0 tx=0x7f1fb400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:32.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.983+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40035f0 con 0x7f1fc414fde0 2024-01-20T08:31:32.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.983+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4003750 con 0x7f1fc414fde0 2024-01-20T08:31:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.983+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4003b00 con 0x7f1fc414fde0 2024-01-20T08:31:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.983+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc40c35c0 con 0x7f1fc414fde0 2024-01-20T08:31:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.983+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc4158cc0 con 0x7f1fc414fde0 2024-01-20T08:31:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:32.984+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc40c3ae0 con 0x7f1fc414fde0 2024-01-20T08:31:33.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.003+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1fb4003c60 con 0x7f1fc414fde0 2024-01-20T08:31:33.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.003+0000 7f1fbaffd640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fa806f330 0x7f1fa80717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:33.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.004+0000 7f1fca413640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fa806f330 0x7f1fa80717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:33.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.004+0000 7f1fca413640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fa806f330 0x7f1fa80717f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009710 tx=0x7f1fc0006c90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:33.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.006+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1fb4057e40 con 0x7f1fc414fde0 2024-01-20T08:31:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.275+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1fb4054180 con 0x7f1fc414fde0 2024-01-20T08:31:33.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.472+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fc40cb300 con 0x7f1fc414fde0 2024-01-20T08:31:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.509+0000 7f1fbaffd640 1 -- 172.21.15.87:0/35932781 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f1fb4017090 con 0x7f1fc414fde0 2024-01-20T08:31:33.516 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:33.516 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fa806f330 msgr2=0x7f1fa80717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fa806f330 0x7f1fa80717f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009710 tx=0x7f1fc0006c90 comp rx=0 tx=0).stop 2024-01-20T08:31:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 msgr2=0x7f1fc4158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 0x7f1fc4158180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c3f0 tx=0x7f1fb400c8b0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 shutdown_connections 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1fa806f330 0x7f1fa80717f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc41590e0 0x7f1fc40c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc414fde0 0x7f1fc4158180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 --2- 172.21.15.87:0/35932781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc40721b0 0x7f1fc41505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 >> 172.21.15.87:0/35932781 conn(0x7f1fc406d510 msgr2=0x7f1fc414e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 shutdown_connections 2024-01-20T08:31:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.510+0000 7f1fcc677640 1 -- 172.21.15.87:0/35932781 wait complete. 2024-01-20T08:31:33.525 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:33.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.620+0000 7f212ac8e640 1 Processor -- start 2024-01-20T08:31:33.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.620+0000 7f212ac8e640 1 -- start start 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.621+0000 7f212ac8e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21240c6860 0x7f21240c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.621+0000 7f212ac8e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240c1590 con 0x7f2124153880 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.621+0000 7f212ac8e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240c2190 con 0x7f21240c1d60 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.621+0000 7f212ac8e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240c7970 con 0x7f21240c6860 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.622+0000 7f2123fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21240c6860 0x7f21240c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.622+0000 7f212922b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f21240c1d60 0x7f212415d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:43372/0 (socket says 172.21.15.87:43372) 2024-01-20T08:31:33.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.622+0000 7f212922b640 1 -- 172.21.15.87:0/1608226557 learned_addr learned my addr 172.21.15.87:0/1608226557 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:33.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.624+0000 7f21237fe640 1 -- 172.21.15.87:0/1608226557 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864461677 0 0) 0x7f21240c2190 con 0x7f21240c1d60 2024-01-20T08:31:33.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.624+0000 7f21237fe640 1 -- 172.21.15.87:0/1608226557 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f210c003520 con 0x7f21240c1d60 2024-01-20T08:31:33.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.625+0000 7f2123fff640 1 -- 172.21.15.87:0/1608226557 >> v1:172.21.15.116:6789/0 conn(0x7f21240c1d60 legacy=0x7f212415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.625+0000 7f2123fff640 1 -- 172.21.15.87:0/1608226557 >> v1:172.21.15.87:6789/0 conn(0x7f2124153880 legacy=0x7f21240c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.625+0000 7f2123fff640 1 -- 172.21.15.87:0/1608226557 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240c7b80 con 0x7f21240c6860 2024-01-20T08:31:33.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.629+0000 7f2123fff640 1 --2- 172.21.15.87:0/1608226557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21240c1820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21180097a0 tx=0x7f211802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8966b4ad481a63 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:33.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.633+0000 7f21237fe640 1 -- 172.21.15.87:0/1608226557 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211802f810 con 0x7f21240c6860 2024-01-20T08:31:33.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.633+0000 7f21237fe640 1 -- 172.21.15.87:0/1608226557 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211802f970 con 0x7f21240c6860 2024-01-20T08:31:33.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.633+0000 7f21237fe640 1 -- 172.21.15.87:0/1608226557 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211802fd00 con 0x7f21240c6860 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 -- 172.21.15.87:0/1608226557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 msgr2=0x7f21240c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 --2- 172.21.15.87:0/1608226557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21240c1820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21180097a0 tx=0x7f211802b9f0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 -- 172.21.15.87:0/1608226557 shutdown_connections 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 --2- 172.21.15.87:0/1608226557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21240c1820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 -- 172.21.15.87:0/1608226557 >> 172.21.15.87:0/1608226557 conn(0x7f2124072610 msgr2=0x7f21240cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 -- 172.21.15.87:0/1608226557 shutdown_connections 2024-01-20T08:31:33.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.634+0000 7f212ac8e640 1 -- 172.21.15.87:0/1608226557 wait complete. 2024-01-20T08:31:33.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.635+0000 7f212ac8e640 1 Processor -- start 2024-01-20T08:31:33.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.635+0000 7f212ac8e640 1 -- start start 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f212ac8e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21240c1d60 0x7f21241e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f212ac8e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21241f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f212ac8e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2124153880 0x7f212415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f212ac8e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21241587e0 con 0x7f21240c6860 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f212ac8e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21241f2270 con 0x7f2124153880 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f212ac8e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2124157a30 con 0x7f21240c1d60 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.636+0000 7f2123fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21241f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.637+0000 7f212922b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2124153880 0x7f212415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.637+0000 7f2123fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21241f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50046/0 (socket says 172.21.15.87:50046) 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.637+0000 7f2123fff640 1 -- 172.21.15.87:0/3072005717 learned_addr learned my addr 172.21.15.87:0/3072005717 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:33.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.637+0000 7f2128a2a640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21240c1d60 0x7f21241e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:33.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.640+0000 7f212922b640 1 -- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21240c1d60 msgr2=0x7f21241e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.640+0000 7f212922b640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21240c1d60 0x7f21241e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.640+0000 7f212922b640 1 -- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 msgr2=0x7f21241f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:33.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.640+0000 7f212922b640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21241f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:33.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.640+0000 7f212922b640 1 -- 172.21.15.87:0/3072005717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2118009400 con 0x7f2124153880 2024-01-20T08:31:33.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.640+0000 7f2123fff640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21241f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:33.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.642+0000 7f212922b640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2124153880 0x7f212415aa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f211c00ca80 tx=0x7f211c00cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.646+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00ac30 con 0x7f2124153880 2024-01-20T08:31:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.646+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c00ad90 con 0x7f2124153880 2024-01-20T08:31:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.646+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c012600 con 0x7f2124153880 2024-01-20T08:31:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.646+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21241f2510 con 0x7f2124153880 2024-01-20T08:31:33.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.646+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2124157c60 con 0x7f2124153880 2024-01-20T08:31:33.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.647+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2124157fa0 con 0x7f2124153880 2024-01-20T08:31:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.889+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f211c007070 con 0x7f2124153880 2024-01-20T08:31:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.889+0000 7f21217fa640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211006af20 0x7f211006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.889+0000 7f2128a2a640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211006af20 0x7f211006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:33.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.890+0000 7f2128a2a640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211006af20 0x7f211006d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21140016e0 tx=0x7f2114009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:33.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.892+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f211c0585f0 con 0x7f2124153880 2024-01-20T08:31:33.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:33.902+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f211c054930 con 0x7f2124153880 2024-01-20T08:31:34.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.074+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2124153c60 con 0x7f2124153880 2024-01-20T08:31:34.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.113+0000 7f21217fa640 1 -- 172.21.15.87:0/3072005717 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f211c017090 con 0x7f2124153880 2024-01-20T08:31:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:34.117 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:34.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211006af20 msgr2=0x7f211006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211006af20 0x7f211006d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21140016e0 tx=0x7f2114009040 comp rx=0 tx=0).stop 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2124153880 msgr2=0x7f212415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2124153880 0x7f212415aa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f211c00ca80 tx=0x7f211c00cf40 comp rx=0 tx=0).stop 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 shutdown_connections 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211006af20 0x7f211006d3e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2124153880 0x7f212415aa30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21240c6860 0x7f21241f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 --2- 172.21.15.87:0/3072005717 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21240c1d60 0x7f21241e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 >> 172.21.15.87:0/3072005717 conn(0x7f2124072610 msgr2=0x7f212415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 shutdown_connections 2024-01-20T08:31:34.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.114+0000 7f212ac8e640 1 -- 172.21.15.87:0/3072005717 wait complete. 2024-01-20T08:31:34.129 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-20T08:31:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.232+0000 7fa7e98ee640 1 Processor -- start 2024-01-20T08:31:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.232+0000 7fa7e98ee640 1 -- start start 2024-01-20T08:31:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.233+0000 7fa7e98ee640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa7e41517f0 0x7fa7e40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.233+0000 7fa7e98ee640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7e40c90e0 0x7fa7e4157460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.233+0000 7fa7e98ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.233+0000 7fa7e98ee640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e40b0520 con 0x7fa7e40c90e0 2024-01-20T08:31:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.233+0000 7fa7e98ee640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e4151bd0 con 0x7fa7e41517f0 2024-01-20T08:31:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.233+0000 7fa7e98ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40ca070 con 0x7fa7e40c9550 2024-01-20T08:31:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.234+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.234+0000 7fa7e2ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa7e41517f0 0x7fa7e40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.234+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:56912/0 (socket says 172.21.15.87:56912) 2024-01-20T08:31:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.234+0000 7fa7e2ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa7e41517f0 0x7fa7e40b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:60964/0 (socket says 172.21.15.87:60964) 2024-01-20T08:31:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.234+0000 7fa7e37fe640 1 -- 172.21.15.87:0/3847457929 learned_addr learned my addr 172.21.15.87:0/3847457929 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:34.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.236+0000 7fa7e37fe640 1 -- 172.21.15.87:0/3847457929 >> v2:172.21.15.116:3300/0 conn(0x7fa7e41517f0 msgr2=0x7fa7e40b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:34.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.236+0000 7fa7e37fe640 1 --2- 172.21.15.87:0/3847457929 >> v2:172.21.15.116:3300/0 conn(0x7fa7e41517f0 0x7fa7e40b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.236+0000 7fa7e37fe640 1 -- 172.21.15.87:0/3847457929 >> v2:172.21.15.87:3300/0 conn(0x7fa7e40c90e0 msgr2=0x7fa7e4157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:34.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.236+0000 7fa7e37fe640 1 --2- 172.21.15.87:0/3847457929 >> v2:172.21.15.87:3300/0 conn(0x7fa7e40c90e0 0x7fa7e4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.236+0000 7fa7e37fe640 1 -- 172.21.15.87:0/3847457929 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e4157ef0 con 0x7fa7e40c9550 2024-01-20T08:31:34.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.239+0000 7fa7e37fe640 1 --2- 172.21.15.87:0/3847457929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40ccd00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa7e406b250 tx=0x7fa7dc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc767c289bcbad5c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e27fc640 1 -- 172.21.15.87:0/3847457929 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc02ca70 con 0x7fa7e40c9550 2024-01-20T08:31:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e27fc640 1 -- 172.21.15.87:0/3847457929 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc02cbd0 con 0x7fa7e40c9550 2024-01-20T08:31:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e27fc640 1 -- 172.21.15.87:0/3847457929 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0343f0 con 0x7fa7e40c9550 2024-01-20T08:31:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 -- 172.21.15.87:0/3847457929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 msgr2=0x7fa7e40ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/3847457929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40ccd00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa7e406b250 tx=0x7fa7dc02bed0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 -- 172.21.15.87:0/3847457929 shutdown_connections 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/3847457929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40ccd00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/3847457929 >> v2:172.21.15.87:3300/0 conn(0x7fa7e40c90e0 0x7fa7e4157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/3847457929 >> v2:172.21.15.116:3300/0 conn(0x7fa7e41517f0 0x7fa7e40b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.243+0000 7fa7e98ee640 1 -- 172.21.15.87:0/3847457929 >> 172.21.15.87:0/3847457929 conn(0x7fa7e406a5b0 msgr2=0x7fa7e406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.244+0000 7fa7e98ee640 1 -- 172.21.15.87:0/3847457929 shutdown_connections 2024-01-20T08:31:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.244+0000 7fa7e98ee640 1 -- 172.21.15.87:0/3847457929 wait complete. 2024-01-20T08:31:34.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.245+0000 7fa7e98ee640 1 Processor -- start 2024-01-20T08:31:34.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.245+0000 7fa7e98ee640 1 -- start start 2024-01-20T08:31:34.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.245+0000 7fa7e98ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7e40c90e0 0x7fa7e4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e98ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e98ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 0x7fa7e414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e98ee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40c58b0 con 0x7fa7e41517f0 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e98ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4152290 con 0x7fa7e40c90e0 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e98ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4154ca0 con 0x7fa7e40c9550 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 0x7fa7e414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 0x7fa7e414bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50060/0 (socket says 172.21.15.87:50060) 2024-01-20T08:31:34.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.246+0000 7fa7e37fe640 1 -- 172.21.15.87:0/632593280 learned_addr learned my addr 172.21.15.87:0/632593280 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:34.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.249+0000 7fa7e37fe640 1 -- 172.21.15.87:0/632593280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 msgr2=0x7fa7e40c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.249+0000 7fa7e37fe640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.250+0000 7fa7e37fe640 1 -- 172.21.15.87:0/632593280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7e40c90e0 msgr2=0x7fa7e4155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.250+0000 7fa7e37fe640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7e40c90e0 0x7fa7e4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.250+0000 7fa7e37fe640 1 -- 172.21.15.87:0/632593280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc009b30 con 0x7fa7e41517f0 2024-01-20T08:31:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.252+0000 7fa7e37fe640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 0x7fa7e414bd10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009e50 tx=0x7fa7dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:34.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.255+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc009510 con 0x7fa7e41517f0 2024-01-20T08:31:34.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.255+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc03e450 con 0x7fa7e41517f0 2024-01-20T08:31:34.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.255+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc03e7e0 con 0x7fa7e41517f0 2024-01-20T08:31:34.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.255+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e40c5260 con 0x7fa7e41517f0 2024-01-20T08:31:34.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.255+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e40c6e50 con 0x7fa7e41517f0 2024-01-20T08:31:34.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.256+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e40c59f0 con 0x7fa7e41517f0 2024-01-20T08:31:34.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.274+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa7dc03e940 con 0x7fa7e41517f0 2024-01-20T08:31:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.275+0000 7fa7e0ff9640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa7c406aed0 0x7fa7c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.275+0000 7fa7e2ffd640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa7c406aed0 0x7fa7c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.276+0000 7fa7e2ffd640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa7c406aed0 0x7fa7c406d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006a90 tx=0x7fa7d4006a20 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:34.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.511+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7dc07b850 con 0x7fa7e41517f0 2024-01-20T08:31:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.511+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa7dc077490 con 0x7fa7e41517f0 2024-01-20T08:31:34.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:34.739+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa7e40c59f0 con 0x7fa7e41517f0 2024-01-20T08:31:35.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.693+0000 7fa7e0ff9640 1 -- 172.21.15.87:0/632593280 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa7e40c59f0 con 0x7fa7e41517f0 2024-01-20T08:31:35.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa7c406aed0 msgr2=0x7fa7c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa7c406aed0 0x7fa7c406d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006a90 tx=0x7fa7d4006a20 comp rx=0 tx=0).stop 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 msgr2=0x7fa7e414bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 0x7fa7e414bd10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009e50 tx=0x7fa7dc004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 shutdown_connections 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa7c406aed0 0x7fa7c406d390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e41517f0 0x7fa7e414bd10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa7e40c9550 0x7fa7e40c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 --2- 172.21.15.87:0/632593280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7e40c90e0 0x7fa7e4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 >> 172.21.15.87:0/632593280 conn(0x7fa7e406a5b0 msgr2=0x7fa7e40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 shutdown_connections 2024-01-20T08:31:35.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.695+0000 7fa7e98ee640 1 -- 172.21.15.87:0/632593280 wait complete. 2024-01-20T08:31:35.711 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300} 2024-01-20T08:31:35.711 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-01-20T08:31:35.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.810+0000 7f8cd1a15640 1 Processor -- start 2024-01-20T08:31:35.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.810+0000 7f8cd1a15640 1 -- start start 2024-01-20T08:31:35.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.811+0000 7f8cd1a15640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ccc06d170 0x7f8ccc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:35.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.811+0000 7f8cd1a15640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc06db00 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.811+0000 7f8cd1a15640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc0719b0 con 0x7f8ccc070e90 2024-01-20T08:31:35.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.811+0000 7f8cd1a15640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c1e90 con 0x7f8ccc06d170 2024-01-20T08:31:35.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.811+0000 7f8ccaffd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8ccc14c3e0 0x7f8ccc06e6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34108/0 (socket says 172.21.15.87:34108) 2024-01-20T08:31:35.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.811+0000 7f8ccaffd640 1 -- 172.21.15.87:0/2424255886 learned_addr learned my addr 172.21.15.87:0/2424255886 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:35.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.812+0000 7f8cca7fc640 1 --2- 172.21.15.87:0/2424255886 >> v2:172.21.15.116:3300/0 conn(0x7f8ccc06d170 0x7f8ccc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:35.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.813+0000 7f8cc9ffb640 1 -- 172.21.15.87:0/2424255886 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329513063 0 0) 0x7f8ccc06db00 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.813+0000 7f8cc9ffb640 1 -- 172.21.15.87:0/2424255886 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb4003520 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.816+0000 7f8cca7fc640 1 -- 172.21.15.87:0/2424255886 >> v1:172.21.15.116:6789/0 conn(0x7f8ccc070e90 legacy=0x7f8ccc0712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.816+0000 7f8cca7fc640 1 -- 172.21.15.87:0/2424255886 >> v1:172.21.15.87:6789/0 conn(0x7f8ccc14c3e0 legacy=0x7f8ccc06e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.816+0000 7f8cca7fc640 1 -- 172.21.15.87:0/2424255886 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0c1030 con 0x7f8ccc06d170 2024-01-20T08:31:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.819+0000 7f8cca7fc640 1 --2- 172.21.15.87:0/2424255886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc06d170 0x7f8ccc06d5c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00097a0 tx=0x7f8cc002ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5007d595a4909667 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:35.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.823+0000 7f8cc9ffb640 1 -- 172.21.15.87:0/2424255886 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc002e830 con 0x7f8ccc06d170 2024-01-20T08:31:35.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.823+0000 7f8cc9ffb640 1 -- 172.21.15.87:0/2424255886 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc002e990 con 0x7f8ccc06d170 2024-01-20T08:31:35.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.823+0000 7f8cc9ffb640 1 -- 172.21.15.87:0/2424255886 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc002ed20 con 0x7f8ccc06d170 2024-01-20T08:31:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 -- 172.21.15.87:0/2424255886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc06d170 msgr2=0x7f8ccc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/2424255886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc06d170 0x7f8ccc06d5c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00097a0 tx=0x7f8cc002ca70 comp rx=0 tx=0).stop 2024-01-20T08:31:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 -- 172.21.15.87:0/2424255886 shutdown_connections 2024-01-20T08:31:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/2424255886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc06d170 0x7f8ccc06d5c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 -- 172.21.15.87:0/2424255886 >> 172.21.15.87:0/2424255886 conn(0x7f8ccc06a9a0 msgr2=0x7f8ccc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 -- 172.21.15.87:0/2424255886 shutdown_connections 2024-01-20T08:31:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.824+0000 7f8cd1a15640 1 -- 172.21.15.87:0/2424255886 wait complete. 2024-01-20T08:31:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.825+0000 7f8cd1a15640 1 Processor -- start 2024-01-20T08:31:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.825+0000 7f8cd1a15640 1 -- start start 2024-01-20T08:31:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.826+0000 7f8cd1a15640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ccc06d170 0x7f8ccc0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.826+0000 7f8cd1a15640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ccc070e90 0x7f8ccc1ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.826+0000 7f8cd1a15640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 0x7f8ccc1eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.826+0000 7f8cd1a15640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0cb630 con 0x7f8ccc06d170 2024-01-20T08:31:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.826+0000 7f8cd1a15640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1ef3d0 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.826+0000 7f8cd1a15640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1eb020 con 0x7f8ccc070e90 2024-01-20T08:31:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.827+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ccc06d170 0x7f8ccc0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.827+0000 7f8ccb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 0x7f8ccc1eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.827+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ccc070e90 0x7f8ccc1ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.827+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ccc06d170 0x7f8ccc0cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37554/0 (socket says 172.21.15.87:37554) 2024-01-20T08:31:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.827+0000 7f8ccb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 0x7f8ccc1eed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49984/0 (socket says 172.21.15.87:49984) 2024-01-20T08:31:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.827+0000 7f8ccaffd640 1 -- 172.21.15.87:0/527768237 learned_addr learned my addr 172.21.15.87:0/527768237 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.830+0000 7f8ccb7fe640 1 -- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ccc070e90 msgr2=0x7f8ccc1ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.830+0000 7f8ccb7fe640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ccc070e90 0x7f8ccc1ebac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.830+0000 7f8ccb7fe640 1 -- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ccc06d170 msgr2=0x7f8ccc0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.830+0000 7f8ccb7fe640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ccc06d170 0x7f8ccc0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.830+0000 7f8ccb7fe640 1 -- 172.21.15.87:0/527768237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009400 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.833+0000 7f8ccb7fe640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 0x7f8ccc1eed40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c390 tx=0x7f8cc400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.837+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40052d0 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.837+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4005430 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.837+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4003830 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.837+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc0cb860 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.837+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1eb5a0 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.837+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc074290 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.855+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8cc4003990 con 0x7f8ccc14c3e0 2024-01-20T08:31:35.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.856+0000 7f8cb3fff640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8cb806af20 0x7f8cb806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:35.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.856+0000 7f8ccaffd640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8cb806af20 0x7f8cb806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:35.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.856+0000 7f8ccaffd640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8cb806af20 0x7f8cb806d3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005c00 tx=0x7f8cbc009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:35.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:35.858+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8cc4057950 con 0x7f8ccc14c3e0 2024-01-20T08:31:36.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.090+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8cc4053c90 con 0x7f8ccc14c3e0 2024-01-20T08:31:36.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.314+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f8ccc071c20 con 0x7f8ccc14c3e0 2024-01-20T08:31:36.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.873+0000 7f8cb3fff640 1 -- 172.21.15.87:0/527768237 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v9) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8cc40170b0 con 0x7f8ccc14c3e0 2024-01-20T08:31:36.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8cb806af20 msgr2=0x7f8cb806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:36.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8cb806af20 0x7f8cb806d3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005c00 tx=0x7f8cbc009040 comp rx=0 tx=0).stop 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 msgr2=0x7f8ccc1eed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 0x7f8ccc1eed40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c390 tx=0x7f8cc400c850 comp rx=0 tx=0).stop 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 shutdown_connections 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8cb806af20 0x7f8cb806d3e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ccc14c3e0 0x7f8ccc1eed40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ccc070e90 0x7f8ccc1ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 --2- 172.21.15.87:0/527768237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ccc06d170 0x7f8ccc0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 >> 172.21.15.87:0/527768237 conn(0x7f8ccc06a9a0 msgr2=0x7f8ccc156370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 shutdown_connections 2024-01-20T08:31:36.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.875+0000 7f8cd1a15640 1 -- 172.21.15.87:0/527768237 wait complete. 2024-01-20T08:31:36.890 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-20T08:31:36.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.991+0000 7fe0f66ec640 1 Processor -- start 2024-01-20T08:31:36.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.991+0000 7fe0f66ec640 1 -- start start 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f66ec640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f0151730 0x7fe0f00c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f66ec640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0f00c8f80 0x7fe0f00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f66ec640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0f00c9fa0 0x7fe0f015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f66ec640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f0151b10 con 0x7fe0f00c9fa0 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f66ec640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f015b7a0 con 0x7fe0f00c8f80 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f66ec640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015b4c0 con 0x7fe0f0151730 2024-01-20T08:31:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.992+0000 7fe0f4c89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0f00c9fa0 0x7fe0f015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:36.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.993+0000 7fe0f4c89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0f00c9fa0 0x7fe0f015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37562/0 (socket says 172.21.15.87:37562) 2024-01-20T08:31:36.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.993+0000 7fe0f4c89640 1 -- 172.21.15.87:0/1755489090 learned_addr learned my addr 172.21.15.87:0/1755489090 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:36.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.995+0000 7fe0f4c89640 1 -- 172.21.15.87:0/1755489090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f0151730 msgr2=0x7fe0f00c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:36.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.995+0000 7fe0f4c89640 1 --2- 172.21.15.87:0/1755489090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f0151730 0x7fe0f00c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:36.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.995+0000 7fe0f4c89640 1 -- 172.21.15.87:0/1755489090 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00c8f80 msgr2=0x7fe0f00c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:36.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.995+0000 7fe0f4c89640 1 --2- 172.21.15.87:0/1755489090 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00c8f80 0x7fe0f00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:36.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.995+0000 7fe0f4c89640 1 -- 172.21.15.87:0/1755489090 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f006ee50 con 0x7fe0f00c9fa0 2024-01-20T08:31:37.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:36.997+0000 7fe0f4c89640 1 --2- 172.21.15.87:0/1755489090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f00c9fa0 0x7fe0f015a930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00cdcf0 tx=0x7fe0e802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da561edf2ebdb54 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.000+0000 7fe0eeffd640 1 -- 172.21.15.87:0/1755489090 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e802cb00 con 0x7fe0f00c9fa0 2024-01-20T08:31:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.000+0000 7fe0eeffd640 1 -- 172.21.15.87:0/1755489090 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e802cc60 con 0x7fe0f00c9fa0 2024-01-20T08:31:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.000+0000 7fe0eeffd640 1 -- 172.21.15.87:0/1755489090 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80343f0 con 0x7fe0f00c9fa0 2024-01-20T08:31:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 -- 172.21.15.87:0/1755489090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f00c9fa0 msgr2=0x7fe0f015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/1755489090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f00c9fa0 0x7fe0f015a930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00cdcf0 tx=0x7fe0e802bf60 comp rx=0 tx=0).stop 2024-01-20T08:31:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 -- 172.21.15.87:0/1755489090 shutdown_connections 2024-01-20T08:31:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/1755489090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f00c9fa0 0x7fe0f015a930 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/1755489090 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00c8f80 0x7fe0f00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/1755489090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f0151730 0x7fe0f00c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 -- 172.21.15.87:0/1755489090 >> 172.21.15.87:0/1755489090 conn(0x7fe0f0070de0 msgr2=0x7fe0f00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 -- 172.21.15.87:0/1755489090 shutdown_connections 2024-01-20T08:31:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.001+0000 7fe0f66ec640 1 -- 172.21.15.87:0/1755489090 wait complete. 2024-01-20T08:31:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.002+0000 7fe0f66ec640 1 Processor -- start 2024-01-20T08:31:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.002+0000 7fe0f66ec640 1 -- start start 2024-01-20T08:31:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.003+0000 7fe0f66ec640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f00c8f80 0x7fe0f006b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.003+0000 7fe0f66ec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f00c9fa0 0x7fe0f01ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.003+0000 7fe0f66ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 0x7fe0f01ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.003+0000 7fe0f66ec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f006ce70 con 0x7fe0f0151730 2024-01-20T08:31:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.003+0000 7fe0f66ec640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f014ab50 con 0x7fe0f00c9fa0 2024-01-20T08:31:37.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.003+0000 7fe0f66ec640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f006c2a0 con 0x7fe0f00c8f80 2024-01-20T08:31:37.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.004+0000 7fe0f4c89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 0x7fe0f01ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:37.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.004+0000 7fe0f4c89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 0x7fe0f01ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37570/0 (socket says 172.21.15.87:37570) 2024-01-20T08:31:37.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.004+0000 7fe0f4c89640 1 -- 172.21.15.87:0/365240647 learned_addr learned my addr 172.21.15.87:0/365240647 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.008+0000 7fe0f4c89640 1 -- 172.21.15.87:0/365240647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f00c8f80 msgr2=0x7fe0f006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.008+0000 7fe0f4c89640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f00c8f80 0x7fe0f006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.008+0000 7fe0f4c89640 1 -- 172.21.15.87:0/365240647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f00c9fa0 msgr2=0x7fe0f01ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.008+0000 7fe0f4c89640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f00c9fa0 0x7fe0f01ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.008+0000 7fe0f4c89640 1 -- 172.21.15.87:0/365240647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8009b30 con 0x7fe0f0151730 2024-01-20T08:31:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.010+0000 7fe0f4c89640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 0x7fe0f01ef8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8036040 tx=0x7fe0e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.013+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e803e450 con 0x7fe0f0151730 2024-01-20T08:31:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.013+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e803e5b0 con 0x7fe0f0151730 2024-01-20T08:31:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.013+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f006c540 con 0x7fe0f0151730 2024-01-20T08:31:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.014+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e803e960 con 0x7fe0f0151730 2024-01-20T08:31:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.014+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f006cb00 con 0x7fe0f0151730 2024-01-20T08:31:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.015+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f00c2630 con 0x7fe0f0151730 2024-01-20T08:31:37.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.259+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe0e803eac0 con 0x7fe0f0151730 2024-01-20T08:31:37.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.260+0000 7fe0ecff9640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0dc06aed0 0x7fe0dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:37.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.260+0000 7fe0effff640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0dc06aed0 0x7fe0dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:37.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.260+0000 7fe0effff640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0dc06aed0 0x7fe0dc06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009710 tx=0x7fe0e0006c90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:37.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.273+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe0e807be10 con 0x7fe0f0151730 2024-01-20T08:31:37.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.273+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe0e8077a50 con 0x7fe0f0151730 2024-01-20T08:31:37.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.478+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe0f0074290 con 0x7fe0f0151730 2024-01-20T08:31:37.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.957+0000 7fe0ecff9640 1 -- 172.21.15.87:0/365240647 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0e80430c0 con 0x7fe0f0151730 2024-01-20T08:31:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0dc06aed0 msgr2=0x7fe0dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0dc06aed0 0x7fe0dc06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009710 tx=0x7fe0e0006c90 comp rx=0 tx=0).stop 2024-01-20T08:31:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 msgr2=0x7fe0f01ef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 0x7fe0f01ef8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8036040 tx=0x7fe0e8004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 shutdown_connections 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0dc06aed0 0x7fe0dc06d390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0f0151730 0x7fe0f01ef8d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f00c9fa0 0x7fe0f01ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 --2- 172.21.15.87:0/365240647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0f00c8f80 0x7fe0f006b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 >> 172.21.15.87:0/365240647 conn(0x7fe0f0070de0 msgr2=0x7fe0f00c1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 shutdown_connections 2024-01-20T08:31:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:37.959+0000 7fe0f66ec640 1 -- 172.21.15.87:0/365240647 wait complete. 2024-01-20T08:31:37.975 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-01-20T08:31:38.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.078+0000 7f63aea3a640 1 Processor -- start 2024-01-20T08:31:38.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.078+0000 7f63aea3a640 1 -- start start 2024-01-20T08:31:38.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.079+0000 7f63aea3a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63a8159d40 0x7f63a80c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:38.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.079+0000 7f63aea3a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a815d8d0 con 0x7f63a8153900 2024-01-20T08:31:38.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.079+0000 7f63aea3a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a80c3c80 con 0x7f63a80c91e0 2024-01-20T08:31:38.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.079+0000 7f63aea3a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a80c8070 con 0x7f63a8159d40 2024-01-20T08:31:38.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.080+0000 7f63acfd7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63a8159d40 0x7f63a80c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:38.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.080+0000 7f63a77fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f63a80c91e0 0x7f63a8159630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38924/0 (socket says 172.21.15.87:38924) 2024-01-20T08:31:38.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.080+0000 7f63a77fe640 1 -- 172.21.15.87:0/1763700465 learned_addr learned my addr 172.21.15.87:0/1763700465 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:38.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.082+0000 7f63a6ffd640 1 -- 172.21.15.87:0/1763700465 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541784059 0 0) 0x7f63a80c3c80 con 0x7f63a80c91e0 2024-01-20T08:31:38.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.082+0000 7f63a6ffd640 1 -- 172.21.15.87:0/1763700465 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6390003520 con 0x7f63a80c91e0 2024-01-20T08:31:38.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.085+0000 7f63acfd7640 1 -- 172.21.15.87:0/1763700465 >> v1:172.21.15.116:6789/0 conn(0x7f63a80c91e0 legacy=0x7f63a8159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:38.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.085+0000 7f63acfd7640 1 -- 172.21.15.87:0/1763700465 >> v1:172.21.15.87:6789/0 conn(0x7f63a8153900 legacy=0x7f63a80c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:38.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.085+0000 7f63acfd7640 1 -- 172.21.15.87:0/1763700465 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a815ccc0 con 0x7f63a8159d40 2024-01-20T08:31:38.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.088+0000 7f63acfd7640 1 --2- 172.21.15.87:0/1763700465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8159d40 0x7f63a80c3740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d814f5a92fc8f2a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:38.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63a6ffd640 1 -- 172.21.15.87:0/1763700465 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a002cd80 con 0x7f63a8159d40 2024-01-20T08:31:38.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63a6ffd640 1 -- 172.21.15.87:0/1763700465 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a0034440 con 0x7f63a8159d40 2024-01-20T08:31:38.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63a6ffd640 1 -- 172.21.15.87:0/1763700465 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a00347d0 con 0x7f63a8159d40 2024-01-20T08:31:38.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 -- 172.21.15.87:0/1763700465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8159d40 msgr2=0x7f63a80c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1763700465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8159d40 0x7f63a80c3740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a002bfd0 comp rx=0 tx=0).stop 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 -- 172.21.15.87:0/1763700465 shutdown_connections 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1763700465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8159d40 0x7f63a80c3740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 -- 172.21.15.87:0/1763700465 >> 172.21.15.87:0/1763700465 conn(0x7f63a8070de0 msgr2=0x7f63a80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 -- 172.21.15.87:0/1763700465 shutdown_connections 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.092+0000 7f63aea3a640 1 -- 172.21.15.87:0/1763700465 wait complete. 2024-01-20T08:31:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.094+0000 7f63aea3a640 1 Processor -- start 2024-01-20T08:31:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.094+0000 7f63aea3a640 1 -- start start 2024-01-20T08:31:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63aea3a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8153900 0x7f63a806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63aea3a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63a8159d40 0x7f63a80c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63aea3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 0x7f63a80c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63aea3a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80c4390 con 0x7f63a8153900 2024-01-20T08:31:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63aea3a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a81f2430 con 0x7f63a814f220 2024-01-20T08:31:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63aea3a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a81f2710 con 0x7f63a8159d40 2024-01-20T08:31:38.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63a77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63a8159d40 0x7f63a80c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:38.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63acfd7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 0x7f63a80c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:38.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63acfd7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 0x7f63a80c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50004/0 (socket says 172.21.15.87:50004) 2024-01-20T08:31:38.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63acfd7640 1 -- 172.21.15.87:0/1289873781 learned_addr learned my addr 172.21.15.87:0/1289873781 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:38.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.095+0000 7f63a77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63a8159d40 0x7f63a80c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36280/0 (socket says 172.21.15.87:36280) 2024-01-20T08:31:38.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.096+0000 7f63a7fff640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8153900 0x7f63a806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:38.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.098+0000 7f63acfd7640 1 -- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63a8159d40 msgr2=0x7f63a80c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:38.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.098+0000 7f63acfd7640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63a8159d40 0x7f63a80c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:38.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.098+0000 7f63acfd7640 1 -- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8153900 msgr2=0x7f63a806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:38.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.098+0000 7f63acfd7640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8153900 0x7f63a806f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:38.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.099+0000 7f63acfd7640 1 -- 172.21.15.87:0/1289873781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0009a70 con 0x7f63a814f220 2024-01-20T08:31:38.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.101+0000 7f63acfd7640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 0x7f63a80c5860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009eb0 tx=0x7f63a0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:38.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.107+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0040650 con 0x7f63a814f220 2024-01-20T08:31:38.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.107+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a00407b0 con 0x7f63a814f220 2024-01-20T08:31:38.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.107+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0040b60 con 0x7f63a814f220 2024-01-20T08:31:38.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.108+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a81f29f0 con 0x7f63a814f220 2024-01-20T08:31:38.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.108+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a81f2fb0 con 0x7f63a814f220 2024-01-20T08:31:38.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.109+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a80c5040 con 0x7f63a814f220 2024-01-20T08:31:38.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.127+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f63a0040cc0 con 0x7f63a814f220 2024-01-20T08:31:38.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.127+0000 7f63a4ff9640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639406af50 0x7f639406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:38.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.127+0000 7f63a7fff640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639406af50 0x7f639406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:38.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.128+0000 7f63a7fff640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639406af50 0x7f639406d410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6398001780 tx=0x7f6398009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:38.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.129+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f63a007b380 con 0x7f63a814f220 2024-01-20T08:31:38.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.392+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f63a00776c0 con 0x7f63a814f220 2024-01-20T08:31:38.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:38.617+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f63a80c5040 con 0x7f63a814f220 2024-01-20T08:31:39.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.041+0000 7f63a4ff9640 1 -- 172.21.15.87:0/1289873781 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v12) v1 ==== 115+0+0 (secure 0 0 0) 0x7f63a80c5040 con 0x7f63a814f220 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639406af50 msgr2=0x7f639406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639406af50 0x7f639406d410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6398001780 tx=0x7f6398009040 comp rx=0 tx=0).stop 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 msgr2=0x7f63a80c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 0x7f63a80c5860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009eb0 tx=0x7f63a0004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 shutdown_connections 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f639406af50 0x7f639406d410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63a814f220 0x7f63a80c5860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63a8159d40 0x7f63a80c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 --2- 172.21.15.87:0/1289873781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63a8153900 0x7f63a806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.043+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 >> 172.21.15.87:0/1289873781 conn(0x7f63a8070de0 msgr2=0x7f63a80cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:39.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.044+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 shutdown_connections 2024-01-20T08:31:39.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.044+0000 7f63aea3a640 1 -- 172.21.15.87:0/1289873781 wait complete. 2024-01-20T08:31:39.058 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-20T08:31:39.058 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:39.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.167+0000 7f8e0e5de640 1 Processor -- start 2024-01-20T08:31:39.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.167+0000 7f8e0e5de640 1 -- start start 2024-01-20T08:31:39.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.168+0000 7f8e0e5de640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8e080ca090 0x7f8e080cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.168+0000 7f8e0e5de640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e0815d6f0 0x7f8e080c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.168+0000 7f8e0e5de640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e08153eb0 con 0x7f8e08153ad0 2024-01-20T08:31:39.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.168+0000 7f8e0e5de640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0815a560 con 0x7f8e080ca090 2024-01-20T08:31:39.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.168+0000 7f8e0e5de640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0815dc80 con 0x7f8e0815d6f0 2024-01-20T08:31:39.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.169+0000 7f8e077fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8e080ca090 0x7f8e080cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.169+0000 7f8e0cb7b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e0815d6f0 0x7f8e080c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.169+0000 7f8e07fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8e08153ad0 0x7f8e080ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38940/0 (socket says 172.21.15.87:38940) 2024-01-20T08:31:39.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.169+0000 7f8e07fff640 1 -- 172.21.15.87:0/2226763366 learned_addr learned my addr 172.21.15.87:0/2226763366 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:39.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.172+0000 7f8e0cb7b640 1 -- 172.21.15.87:0/2226763366 >> v2:172.21.15.87:3300/0 conn(0x7f8e080ca090 msgr2=0x7f8e080cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:39.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.172+0000 7f8e0cb7b640 1 --2- 172.21.15.87:0/2226763366 >> v2:172.21.15.87:3300/0 conn(0x7f8e080ca090 0x7f8e080cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.172+0000 7f8e0cb7b640 1 -- 172.21.15.87:0/2226763366 >> v1:172.21.15.116:6789/0 conn(0x7f8e08153ad0 legacy=0x7f8e080ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.172+0000 7f8e0cb7b640 1 -- 172.21.15.87:0/2226763366 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080cb760 con 0x7f8e0815d6f0 2024-01-20T08:31:39.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.177+0000 7f8e0cb7b640 1 --2- 172.21.15.87:0/2226763366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e0815d6f0 0x7f8e080c3e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e080ca8a0 tx=0x7f8e0002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d227f49931d0f86 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.180+0000 7f8e06ffd640 1 -- 172.21.15.87:0/2226763366 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0002cc80 con 0x7f8e0815d6f0 2024-01-20T08:31:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.180+0000 7f8e06ffd640 1 -- 172.21.15.87:0/2226763366 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0002cde0 con 0x7f8e0815d6f0 2024-01-20T08:31:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.180+0000 7f8e06ffd640 1 -- 172.21.15.87:0/2226763366 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00034660 con 0x7f8e0815d6f0 2024-01-20T08:31:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2226763366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e0815d6f0 msgr2=0x7f8e080c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2226763366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e0815d6f0 0x7f8e080c3e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e080ca8a0 tx=0x7f8e0002bed0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2226763366 shutdown_connections 2024-01-20T08:31:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2226763366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e0815d6f0 0x7f8e080c3e60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2226763366 >> v2:172.21.15.87:3300/0 conn(0x7f8e080ca090 0x7f8e080cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2226763366 >> 172.21.15.87:0/2226763366 conn(0x7f8e0808af90 msgr2=0x7f8e0808b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2226763366 shutdown_connections 2024-01-20T08:31:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.181+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2226763366 wait complete. 2024-01-20T08:31:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.182+0000 7f8e0e5de640 1 Processor -- start 2024-01-20T08:31:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 -- start start 2024-01-20T08:31:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 0x7f8e0815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e08153ad0 0x7f8e081497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e0815d6f0 0x7f8e08149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0814a240 con 0x7f8e0815d6f0 2024-01-20T08:31:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0814dc20 con 0x7f8e080ca090 2024-01-20T08:31:39.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.183+0000 7f8e0e5de640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0814c1c0 con 0x7f8e08153ad0 2024-01-20T08:31:39.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.184+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 0x7f8e0815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.184+0000 7f8e077fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e08153ad0 0x7f8e081497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.184+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 0x7f8e0815c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50020/0 (socket says 172.21.15.87:50020) 2024-01-20T08:31:39.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.184+0000 7f8e07fff640 1 -- 172.21.15.87:0/2758328651 learned_addr learned my addr 172.21.15.87:0/2758328651 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:39.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.184+0000 7f8e0cb7b640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e0815d6f0 0x7f8e08149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.188+0000 7f8e07fff640 1 -- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e08153ad0 msgr2=0x7f8e081497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.188+0000 7f8e07fff640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e08153ad0 0x7f8e081497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.188+0000 7f8e07fff640 1 -- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e0815d6f0 msgr2=0x7f8e08149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.188+0000 7f8e07fff640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e0815d6f0 0x7f8e08149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.188+0000 7f8e07fff640 1 -- 172.21.15.87:0/2758328651 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00009a70 con 0x7f8e080ca090 2024-01-20T08:31:39.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.190+0000 7f8e07fff640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 0x7f8e0815c370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c400 tx=0x7f8df800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:39.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.194+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8005ce0 con 0x7f8e080ca090 2024-01-20T08:31:39.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.194+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8005e40 con 0x7f8e080ca090 2024-01-20T08:31:39.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.194+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0814a7f0 con 0x7f8e080ca090 2024-01-20T08:31:39.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.194+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80072d0 con 0x7f8e080ca090 2024-01-20T08:31:39.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.194+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0814ccd0 con 0x7f8e080ca090 2024-01-20T08:31:39.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.195+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e08074290 con 0x7f8e080ca090 2024-01-20T08:31:39.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.215+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8df8007430 con 0x7f8e080ca090 2024-01-20T08:31:39.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.215+0000 7f8e04ff9640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8df406af50 0x7f8df406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.215+0000 7f8e077fe640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8df406af50 0x7f8df406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.216+0000 7f8e077fe640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8df406af50 0x7f8df406d410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e080c3f40 tx=0x7f8dfc009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:39.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.217+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8df8057a80 con 0x7f8e080ca090 2024-01-20T08:31:39.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.455+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8df8053dc0 con 0x7f8e080ca090 2024-01-20T08:31:39.475 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:31:39.477 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:31:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.672+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e08153eb0 con 0x7f8e080ca090 2024-01-20T08:31:39.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.685+0000 7f8e04ff9640 1 -- 172.21.15.87:0/2758328651 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6553 (secure 0 0 0) 0x7f8e08153eb0 con 0x7f8e080ca090 2024-01-20T08:31:39.688 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:39.689 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:38.972628+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2],"up":{"mds_0":4293,"mds_1":4290,"mds_2":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":2,"incarnation":13,"state":"up:creating","state_seq":1,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4290":{"gid":4290,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-20T08:31:39.689 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 13 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8df406af50 msgr2=0x7f8df406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8df406af50 0x7f8df406d410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e080c3f40 tx=0x7f8dfc009190 comp rx=0 tx=0).stop 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 msgr2=0x7f8e0815c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 0x7f8e0815c370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c400 tx=0x7f8df800c8c0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 shutdown_connections 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8df406af50 0x7f8df406d410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e0815d6f0 0x7f8e08149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e08153ad0 0x7f8e081497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 --2- 172.21.15.87:0/2758328651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e080ca090 0x7f8e0815c370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 >> 172.21.15.87:0/2758328651 conn(0x7f8e0808af90 msgr2=0x7f8e080cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 shutdown_connections 2024-01-20T08:31:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.687+0000 7f8e0e5de640 1 -- 172.21.15.87:0/2758328651 wait complete. 2024-01-20T08:31:39.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:39.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.804+0000 7ff6ae59b640 1 Processor -- start 2024-01-20T08:31:39.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.804+0000 7ff6ae59b640 1 -- start start 2024-01-20T08:31:39.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.805+0000 7ff6ae59b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6a8070cf0 0x7ff6a80c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.805+0000 7ff6ae59b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 0x7ff6a8157b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.805+0000 7ff6ae59b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a815ba40 con 0x7ff6a80ca260 2024-01-20T08:31:39.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.805+0000 7ff6ae59b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a80c3bb0 con 0x7ff6a8070cf0 2024-01-20T08:31:39.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.805+0000 7ff6ae59b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80c96f0 con 0x7ff6a80710d0 2024-01-20T08:31:39.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.805+0000 7ff6a77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 0x7ff6a8157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.806+0000 7ff6a77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 0x7ff6a8157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36294/0 (socket says 172.21.15.87:36294) 2024-01-20T08:31:39.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.806+0000 7ff6a77fe640 1 -- 172.21.15.87:0/4278676562 learned_addr learned my addr 172.21.15.87:0/4278676562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:39.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.808+0000 7ff6a77fe640 1 -- 172.21.15.87:0/4278676562 >> v2:172.21.15.87:3300/0 conn(0x7ff6a8070cf0 msgr2=0x7ff6a80c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:39.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.808+0000 7ff6a77fe640 1 --2- 172.21.15.87:0/4278676562 >> v2:172.21.15.87:3300/0 conn(0x7ff6a8070cf0 0x7ff6a80c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.808+0000 7ff6a77fe640 1 -- 172.21.15.87:0/4278676562 >> v1:172.21.15.87:6789/0 conn(0x7ff6a80ca260 legacy=0x7ff6a8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:39.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.808+0000 7ff6a77fe640 1 -- 172.21.15.87:0/4278676562 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a81576e0 con 0x7ff6a80710d0 2024-01-20T08:31:39.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.810+0000 7ff6a77fe640 1 --2- 172.21.15.87:0/4278676562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 0x7ff6a8157b70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009870 tx=0x7ff69c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc1217d79f724123 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.813+0000 7ff6a6ffd640 1 -- 172.21.15.87:0/4278676562 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c030840 con 0x7ff6a80710d0 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6a6ffd640 1 -- 172.21.15.87:0/4278676562 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c0309a0 con 0x7ff6a80710d0 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6a6ffd640 1 -- 172.21.15.87:0/4278676562 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c030d30 con 0x7ff6a80710d0 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 -- 172.21.15.87:0/4278676562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 msgr2=0x7ff6a8157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/4278676562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 0x7ff6a8157b70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009870 tx=0x7ff69c02b980 comp rx=0 tx=0).stop 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 -- 172.21.15.87:0/4278676562 shutdown_connections 2024-01-20T08:31:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/4278676562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80710d0 0x7ff6a8157b70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/4278676562 >> v2:172.21.15.87:3300/0 conn(0x7ff6a8070cf0 0x7ff6a80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 -- 172.21.15.87:0/4278676562 >> 172.21.15.87:0/4278676562 conn(0x7ff6a8066ac0 msgr2=0x7ff6a806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 -- 172.21.15.87:0/4278676562 shutdown_connections 2024-01-20T08:31:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.814+0000 7ff6ae59b640 1 -- 172.21.15.87:0/4278676562 wait complete. 2024-01-20T08:31:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.815+0000 7ff6ae59b640 1 Processor -- start 2024-01-20T08:31:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.815+0000 7ff6ae59b640 1 -- start start 2024-01-20T08:31:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.816+0000 7ff6ae59b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6a80710d0 0x7ff6a8150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.816+0000 7ff6ae59b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 0x7ff6a81f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.816+0000 7ff6ae59b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a8150380 0x7ff6a80c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.816+0000 7ff6ae59b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81e4870 con 0x7ff6a8150380 2024-01-20T08:31:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.816+0000 7ff6ae59b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81e12b0 con 0x7ff6a80710d0 2024-01-20T08:31:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.816+0000 7ff6ae59b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81f38d0 con 0x7ff6a80ca260 2024-01-20T08:31:39.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.817+0000 7ff6a77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 0x7ff6a81f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.817+0000 7ff6a7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6a80710d0 0x7ff6a8150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:39.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.817+0000 7ff6a77fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 0x7ff6a81f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36306/0 (socket says 172.21.15.87:36306) 2024-01-20T08:31:39.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.817+0000 7ff6a77fe640 1 -- 172.21.15.87:0/2174312366 learned_addr learned my addr 172.21.15.87:0/2174312366 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.820+0000 7ff6a77fe640 1 -- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6a80710d0 msgr2=0x7ff6a8150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.820+0000 7ff6a77fe640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6a80710d0 0x7ff6a8150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.820+0000 7ff6a77fe640 1 -- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a8150380 msgr2=0x7ff6a80c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.820+0000 7ff6a77fe640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a8150380 0x7ff6a80c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.820+0000 7ff6a77fe640 1 -- 172.21.15.87:0/2174312366 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0094d0 con 0x7ff6a80ca260 2024-01-20T08:31:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.822+0000 7ff6a77fe640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 0x7ff6a81f3390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009870 tx=0x7ff69c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.826+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c039d60 con 0x7ff6a80ca260 2024-01-20T08:31:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.826+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c043400 con 0x7ff6a80ca260 2024-01-20T08:31:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.826+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a8151780 con 0x7ff6a80ca260 2024-01-20T08:31:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.826+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0437b0 con 0x7ff6a80ca260 2024-01-20T08:31:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.826+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a8151c90 con 0x7ff6a80ca260 2024-01-20T08:31:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:39.827+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a8151a10 con 0x7ff6a80ca260 2024-01-20T08:31:40.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.071+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff69c043910 con 0x7ff6a80ca260 2024-01-20T08:31:40.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.071+0000 7ff6a4ff9640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff69406af20 0x7ff69406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.071+0000 7ff6a7fff640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff69406af20 0x7ff69406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.072+0000 7ff6a7fff640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff69406af20 0x7ff69406d3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff698009990 tx=0x7ff698009110 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:40.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.075+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff69c03e070 con 0x7ff6a80ca260 2024-01-20T08:31:40.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.086+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff69c0780a0 con 0x7ff6a80ca260 2024-01-20T08:31:40.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.289+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a80cda90 con 0x7ff6a80ca260 2024-01-20T08:31:40.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.336+0000 7ff6a4ff9640 1 -- 172.21.15.87:0/2174312366 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7ff69c034090 con 0x7ff6a80ca260 2024-01-20T08:31:40.339 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:40.339 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:16.812221+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:40.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff69406af20 msgr2=0x7ff69406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff69406af20 0x7ff69406d3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff698009990 tx=0x7ff698009110 comp rx=0 tx=0).stop 2024-01-20T08:31:40.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 msgr2=0x7ff6a81f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 0x7ff6a81f3390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009870 tx=0x7ff69c004730 comp rx=0 tx=0).stop 2024-01-20T08:31:40.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 shutdown_connections 2024-01-20T08:31:40.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff69406af20 0x7ff69406d3e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a8150380 0x7ff6a80c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6a80ca260 0x7ff6a81f3390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 --2- 172.21.15.87:0/2174312366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6a80710d0 0x7ff6a8150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 >> 172.21.15.87:0/2174312366 conn(0x7ff6a8070cf0 msgr2=0x7ff6a80c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:40.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 shutdown_connections 2024-01-20T08:31:40.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.338+0000 7ff6ae59b640 1 -- 172.21.15.87:0/2174312366 wait complete. 2024-01-20T08:31:40.353 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-20T08:31:40.353 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:31:40.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.450+0000 7f161bd89640 1 Processor -- start 2024-01-20T08:31:40.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.450+0000 7f161bd89640 1 -- start start 2024-01-20T08:31:40.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f161bd89640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16141513d0 0x7f1614072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f161bd89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614159020 0x7f1614072890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f161bd89640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140c6c30 con 0x7f1614070670 2024-01-20T08:31:40.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f161bd89640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16141581b0 con 0x7f16141513d0 2024-01-20T08:31:40.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f161bd89640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1614157530 con 0x7f1614159020 2024-01-20T08:31:40.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f161a326640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614159020 0x7f1614072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f1619324640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1614070670 0x7f1614158910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34136/0 (socket says 172.21.15.87:34136) 2024-01-20T08:31:40.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.451+0000 7f1619324640 1 -- 172.21.15.87:0/3968181095 learned_addr learned my addr 172.21.15.87:0/3968181095 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:40.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.452+0000 7f1619b25640 1 --2- 172.21.15.87:0/3968181095 >> v2:172.21.15.116:3300/0 conn(0x7f16141513d0 0x7f1614072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.454+0000 7f1619b25640 1 -- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614159020 msgr2=0x7f1614072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:40.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.454+0000 7f1619b25640 1 --2- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614159020 0x7f1614072890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.454+0000 7f1619b25640 1 -- 172.21.15.87:0/3968181095 >> v1:172.21.15.87:6789/0 conn(0x7f1614070670 legacy=0x7f1614158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.454+0000 7f1619b25640 1 -- 172.21.15.87:0/3968181095 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614157810 con 0x7f16141513d0 2024-01-20T08:31:40.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.456+0000 7f1619b25640 1 --2- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f1614072350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f16100097a0 tx=0x7f161002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84607ed424cd789c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:40.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.460+0000 7f1618b23640 1 -- 172.21.15.87:0/3968181095 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610030710 con 0x7f16141513d0 2024-01-20T08:31:40.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.460+0000 7f1618b23640 1 -- 172.21.15.87:0/3968181095 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161002e050 con 0x7f16141513d0 2024-01-20T08:31:40.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.460+0000 7f1618b23640 1 -- 172.21.15.87:0/3968181095 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610030a90 con 0x7f16141513d0 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.460+0000 7f161bd89640 1 -- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 msgr2=0x7f1614072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.460+0000 7f161bd89640 1 --2- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f1614072350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f16100097a0 tx=0x7f161002b790 comp rx=0 tx=0).stop 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.461+0000 7f161bd89640 1 -- 172.21.15.87:0/3968181095 shutdown_connections 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.461+0000 7f161bd89640 1 --2- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614159020 0x7f1614072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.461+0000 7f161bd89640 1 --2- 172.21.15.87:0/3968181095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f1614072350 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.461+0000 7f161bd89640 1 -- 172.21.15.87:0/3968181095 >> 172.21.15.87:0/3968181095 conn(0x7f161406dd80 msgr2=0x7f161406e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.461+0000 7f161bd89640 1 -- 172.21.15.87:0/3968181095 shutdown_connections 2024-01-20T08:31:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.461+0000 7f161bd89640 1 -- 172.21.15.87:0/3968181095 wait complete. 2024-01-20T08:31:40.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 Processor -- start 2024-01-20T08:31:40.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 -- start start 2024-01-20T08:31:40.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614070670 0x7f16141f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f161414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 0x7f1614145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161415b420 con 0x7f1614159020 2024-01-20T08:31:40.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16140ca3b0 con 0x7f16141513d0 2024-01-20T08:31:40.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.462+0000 7f161bd89640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161414f8d0 con 0x7f1614070670 2024-01-20T08:31:40.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.463+0000 7f161a326640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 0x7f1614145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.463+0000 7f1619b25640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614070670 0x7f16141f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.463+0000 7f1619324640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f161414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.463+0000 7f161a326640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 0x7f1614145960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37628/0 (socket says 172.21.15.87:37628) 2024-01-20T08:31:40.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.463+0000 7f161a326640 1 -- 172.21.15.87:0/1130601835 learned_addr learned my addr 172.21.15.87:0/1130601835 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:40.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.467+0000 7f161a326640 1 -- 172.21.15.87:0/1130601835 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614070670 msgr2=0x7f16141f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.467+0000 7f161a326640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614070670 0x7f16141f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.467+0000 7f161a326640 1 -- 172.21.15.87:0/1130601835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 msgr2=0x7f161414fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.467+0000 7f161a326640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f161414fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.467+0000 7f161a326640 1 -- 172.21.15.87:0/1130601835 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610009400 con 0x7f1614159020 2024-01-20T08:31:40.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.469+0000 7f161a326640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 0x7f1614145960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f160c00efc0 tx=0x7f160c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:40.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.473+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c009b20 con 0x7f1614159020 2024-01-20T08:31:40.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.473+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c00f040 con 0x7f1614159020 2024-01-20T08:31:40.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.473+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0146c0 con 0x7f1614159020 2024-01-20T08:31:40.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.473+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16141f13f0 con 0x7f1614159020 2024-01-20T08:31:40.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.473+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16141597f0 con 0x7f1614159020 2024-01-20T08:31:40.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.474+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161414fa10 con 0x7f1614159020 2024-01-20T08:31:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.492+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f160c014820 con 0x7f1614159020 2024-01-20T08:31:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.492+0000 7f160a7fc640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f15f806af00 0x7f15f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.492+0000 7f1619b25640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f15f806af00 0x7f15f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.493+0000 7f1619b25640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f15f806af00 0x7f15f806d3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1610034000 tx=0x7f161002f000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:40.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.731+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f160c059560 con 0x7f1614159020 2024-01-20T08:31:40.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.731+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f160c0558a0 con 0x7f1614159020 2024-01-20T08:31:40.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.899+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1614157dd0 con 0x7f15f806af00 2024-01-20T08:31:40.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.908+0000 7f160a7fc640 1 -- 172.21.15.87:0/1130601835 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345253 (secure 0 0 0) 0x7f1610040070 con 0x7f15f806af00 2024-01-20T08:31:40.916 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:40.918 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f15f806af00 msgr2=0x7f15f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f15f806af00 0x7f15f806d3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1610034000 tx=0x7f161002f000 comp rx=0 tx=0).stop 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 msgr2=0x7f1614145960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 0x7f1614145960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f160c00efc0 tx=0x7f160c00c490 comp rx=0 tx=0).stop 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 shutdown_connections 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f15f806af00 0x7f15f806d3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614159020 0x7f1614145960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16141513d0 0x7f161414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 --2- 172.21.15.87:0/1130601835 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1614070670 0x7f16141f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 >> 172.21.15.87:0/1130601835 conn(0x7f161406dd80 msgr2=0x7f1614072320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:40.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 shutdown_connections 2024-01-20T08:31:40.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:40.911+0000 7f161bd89640 1 -- 172.21.15.87:0/1130601835 wait complete. 2024-01-20T08:31:40.925 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2024-01-20T08:31:40.283865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595611,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":225,"num_write_kb":2443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219456,"kb_used_data":3280,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755264,"statfs":{"total":773094113280,"available":772869390336,"internally_reserved":0,"allocated":3358720,"data_stored":2022322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":745,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002807"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147627+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:31:16.147627+0000","last_peered":"2024-01-20T08:31:16.147627+0000","last_clean":"2024-01-20T08:31:16.147627+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:31:16.147627+0000","last_undegraded":"2024-01-20T08:31:16.147627+0000","last_fullsized":"2024-01-20T08:31:16.147627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159832+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:31:16.159832+0000","last_peered":"2024-01-20T08:31:16.159832+0000","last_clean":"2024-01-20T08:31:16.159832+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:31:16.159832+0000","last_undegraded":"2024-01-20T08:31:16.159832+0000","last_fullsized":"2024-01-20T08:31:16.159832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375104+0000","last_change":"2024-01-20T08:31:16.175747+0000","last_active":"2024-01-20T08:31:17.375104+0000","last_peered":"2024-01-20T08:31:17.375104+0000","last_clean":"2024-01-20T08:31:17.375104+0000","last_became_active":"2024-01-20T08:31:16.174647+0000","last_became_peered":"2024-01-20T08:31:16.174647+0000","last_unstale":"2024-01-20T08:31:17.375104+0000","last_undegraded":"2024-01-20T08:31:17.375104+0000","last_fullsized":"2024-01-20T08:31:17.375104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:26:34.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159915+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:31:16.159915+0000","last_peered":"2024-01-20T08:31:16.159915+0000","last_clean":"2024-01-20T08:31:16.159915+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:31:16.159915+0000","last_undegraded":"2024-01-20T08:31:16.159915+0000","last_fullsized":"2024-01-20T08:31:16.159915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159743+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:31:16.159743+0000","last_peered":"2024-01-20T08:31:16.159743+0000","last_clean":"2024-01-20T08:31:16.159743+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:31:16.159743+0000","last_undegraded":"2024-01-20T08:31:16.159743+0000","last_fullsized":"2024-01-20T08:31:16.159743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373746+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:31:17.373746+0000","last_peered":"2024-01-20T08:31:17.373746+0000","last_clean":"2024-01-20T08:31:17.373746+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:31:17.373746+0000","last_undegraded":"2024-01-20T08:31:17.373746+0000","last_fullsized":"2024-01-20T08:31:17.373746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087496+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:31:16.087496+0000","last_peered":"2024-01-20T08:31:16.087496+0000","last_clean":"2024-01-20T08:31:16.087496+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:31:16.087496+0000","last_undegraded":"2024-01-20T08:31:16.087496+0000","last_fullsized":"2024-01-20T08:31:16.087496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.171846+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:31:16.171846+0000","last_peered":"2024-01-20T08:31:16.171846+0000","last_clean":"2024-01-20T08:31:16.171846+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:31:16.171846+0000","last_undegraded":"2024-01-20T08:31:16.171846+0000","last_fullsized":"2024-01-20T08:31:16.171846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374782+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:31:17.374782+0000","last_peered":"2024-01-20T08:31:17.374782+0000","last_clean":"2024-01-20T08:31:17.374782+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:31:17.374782+0000","last_undegraded":"2024-01-20T08:31:17.374782+0000","last_fullsized":"2024-01-20T08:31:17.374782+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087803+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:31:16.087803+0000","last_peered":"2024-01-20T08:31:16.087803+0000","last_clean":"2024-01-20T08:31:16.087803+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:31:16.087803+0000","last_undegraded":"2024-01-20T08:31:16.087803+0000","last_fullsized":"2024-01-20T08:31:16.087803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152170+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:31:16.152170+0000","last_peered":"2024-01-20T08:31:16.152170+0000","last_clean":"2024-01-20T08:31:16.152170+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:31:16.152170+0000","last_undegraded":"2024-01-20T08:31:16.152170+0000","last_fullsized":"2024-01-20T08:31:16.152170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374153+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:31:17.374153+0000","last_peered":"2024-01-20T08:31:17.374153+0000","last_clean":"2024-01-20T08:31:17.374153+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:31:17.374153+0000","last_undegraded":"2024-01-20T08:31:17.374153+0000","last_fullsized":"2024-01-20T08:31:17.374153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152179+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:31:16.152179+0000","last_peered":"2024-01-20T08:31:16.152179+0000","last_clean":"2024-01-20T08:31:16.152179+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:31:16.152179+0000","last_undegraded":"2024-01-20T08:31:16.152179+0000","last_fullsized":"2024-01-20T08:31:16.152179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999492+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:31:37.999492+0000","last_peered":"2024-01-20T08:31:37.999492+0000","last_clean":"2024-01-20T08:31:37.999492+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:31:37.999492+0000","last_undegraded":"2024-01-20T08:31:37.999492+0000","last_fullsized":"2024-01-20T08:31:37.999492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137443+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:31:17.137443+0000","last_peered":"2024-01-20T08:31:17.137443+0000","last_clean":"2024-01-20T08:31:17.137443+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:31:17.137443+0000","last_undegraded":"2024-01-20T08:31:17.137443+0000","last_fullsized":"2024-01-20T08:31:17.137443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152379+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:31:16.152379+0000","last_peered":"2024-01-20T08:31:16.152379+0000","last_clean":"2024-01-20T08:31:16.152379+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:31:16.152379+0000","last_undegraded":"2024-01-20T08:31:16.152379+0000","last_fullsized":"2024-01-20T08:31:16.152379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090103+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:31:16.090103+0000","last_peered":"2024-01-20T08:31:16.090103+0000","last_clean":"2024-01-20T08:31:16.090103+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:31:16.090103+0000","last_undegraded":"2024-01-20T08:31:16.090103+0000","last_fullsized":"2024-01-20T08:31:16.090103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.188856+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:31:17.188856+0000","last_peered":"2024-01-20T08:31:17.188856+0000","last_clean":"2024-01-20T08:31:17.188856+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:31:17.188856+0000","last_undegraded":"2024-01-20T08:31:17.188856+0000","last_fullsized":"2024-01-20T08:31:17.188856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147252+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:31:16.147252+0000","last_peered":"2024-01-20T08:31:16.147252+0000","last_clean":"2024-01-20T08:31:16.147252+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:31:16.147252+0000","last_undegraded":"2024-01-20T08:31:16.147252+0000","last_fullsized":"2024-01-20T08:31:16.147252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147399+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:31:16.147399+0000","last_peered":"2024-01-20T08:31:16.147399+0000","last_clean":"2024-01-20T08:31:16.147399+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:31:16.147399+0000","last_undegraded":"2024-01-20T08:31:16.147399+0000","last_fullsized":"2024-01-20T08:31:16.147399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374822+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:31:17.374822+0000","last_peered":"2024-01-20T08:31:17.374822+0000","last_clean":"2024-01-20T08:31:17.374822+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:31:17.374822+0000","last_undegraded":"2024-01-20T08:31:17.374822+0000","last_fullsized":"2024-01-20T08:31:17.374822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151575+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:31:16.151575+0000","last_peered":"2024-01-20T08:31:16.151575+0000","last_clean":"2024-01-20T08:31:16.151575+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:31:16.151575+0000","last_undegraded":"2024-01-20T08:31:16.151575+0000","last_fullsized":"2024-01-20T08:31:16.151575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087150+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:31:16.087150+0000","last_peered":"2024-01-20T08:31:16.087150+0000","last_clean":"2024-01-20T08:31:16.087150+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:31:16.087150+0000","last_undegraded":"2024-01-20T08:31:16.087150+0000","last_fullsized":"2024-01-20T08:31:16.087150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374458+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:31:17.374458+0000","last_peered":"2024-01-20T08:31:17.374458+0000","last_clean":"2024-01-20T08:31:17.374458+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:31:17.374458+0000","last_undegraded":"2024-01-20T08:31:17.374458+0000","last_fullsized":"2024-01-20T08:31:17.374458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147691+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:31:16.147691+0000","last_peered":"2024-01-20T08:31:16.147691+0000","last_clean":"2024-01-20T08:31:16.147691+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:31:16.147691+0000","last_undegraded":"2024-01-20T08:31:16.147691+0000","last_fullsized":"2024-01-20T08:31:16.147691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.998883+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:31:37.998883+0000","last_peered":"2024-01-20T08:31:37.998883+0000","last_clean":"2024-01-20T08:31:37.998883+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:31:37.998883+0000","last_undegraded":"2024-01-20T08:31:37.998883+0000","last_fullsized":"2024-01-20T08:31:37.998883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374019+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:31:17.374019+0000","last_peered":"2024-01-20T08:31:17.374019+0000","last_clean":"2024-01-20T08:31:17.374019+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:31:17.374019+0000","last_undegraded":"2024-01-20T08:31:17.374019+0000","last_fullsized":"2024-01-20T08:31:17.374019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147548+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:31:16.147548+0000","last_peered":"2024-01-20T08:31:16.147548+0000","last_clean":"2024-01-20T08:31:16.147548+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:31:16.147548+0000","last_undegraded":"2024-01-20T08:31:16.147548+0000","last_fullsized":"2024-01-20T08:31:16.147548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.293427+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:31:16.293427+0000","last_peered":"2024-01-20T08:31:16.293427+0000","last_clean":"2024-01-20T08:31:16.293427+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:31:16.293427+0000","last_undegraded":"2024-01-20T08:31:16.293427+0000","last_fullsized":"2024-01-20T08:31:16.293427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373815+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:31:17.373815+0000","last_peered":"2024-01-20T08:31:17.373815+0000","last_clean":"2024-01-20T08:31:17.373815+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:31:17.373815+0000","last_undegraded":"2024-01-20T08:31:17.373815+0000","last_fullsized":"2024-01-20T08:31:17.373815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090530+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:31:16.090530+0000","last_peered":"2024-01-20T08:31:16.090530+0000","last_clean":"2024-01-20T08:31:16.090530+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:31:16.090530+0000","last_undegraded":"2024-01-20T08:31:16.090530+0000","last_fullsized":"2024-01-20T08:31:16.090530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137361+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:31:17.137361+0000","last_peered":"2024-01-20T08:31:17.137361+0000","last_clean":"2024-01-20T08:31:17.137361+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:31:17.137361+0000","last_undegraded":"2024-01-20T08:31:17.137361+0000","last_fullsized":"2024-01-20T08:31:17.137361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098675+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:31:17.098675+0000","last_peered":"2024-01-20T08:31:17.098675+0000","last_clean":"2024-01-20T08:31:17.098675+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:31:17.098675+0000","last_undegraded":"2024-01-20T08:31:17.098675+0000","last_fullsized":"2024-01-20T08:31:17.098675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169805+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:31:16.169805+0000","last_peered":"2024-01-20T08:31:16.169805+0000","last_clean":"2024-01-20T08:31:16.169805+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:31:16.169805+0000","last_undegraded":"2024-01-20T08:31:16.169805+0000","last_fullsized":"2024-01-20T08:31:16.169805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159306+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:31:16.159306+0000","last_peered":"2024-01-20T08:31:16.159306+0000","last_clean":"2024-01-20T08:31:16.159306+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:31:16.159306+0000","last_undegraded":"2024-01-20T08:31:16.159306+0000","last_fullsized":"2024-01-20T08:31:16.159306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098613+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:31:17.098613+0000","last_peered":"2024-01-20T08:31:17.098613+0000","last_clean":"2024-01-20T08:31:17.098613+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:31:17.098613+0000","last_undegraded":"2024-01-20T08:31:17.098613+0000","last_fullsized":"2024-01-20T08:31:17.098613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087311+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:31:16.087311+0000","last_peered":"2024-01-20T08:31:16.087311+0000","last_clean":"2024-01-20T08:31:16.087311+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:31:16.087311+0000","last_undegraded":"2024-01-20T08:31:16.087311+0000","last_fullsized":"2024-01-20T08:31:16.087311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152112+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:31:16.152112+0000","last_peered":"2024-01-20T08:31:16.152112+0000","last_clean":"2024-01-20T08:31:16.152112+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:31:16.152112+0000","last_undegraded":"2024-01-20T08:31:16.152112+0000","last_fullsized":"2024-01-20T08:31:16.152112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098960+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:31:17.098960+0000","last_peered":"2024-01-20T08:31:17.098960+0000","last_clean":"2024-01-20T08:31:17.098960+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:31:17.098960+0000","last_undegraded":"2024-01-20T08:31:17.098960+0000","last_fullsized":"2024-01-20T08:31:17.098960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159100+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:31:16.159100+0000","last_peered":"2024-01-20T08:31:16.159100+0000","last_clean":"2024-01-20T08:31:16.159100+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:31:16.159100+0000","last_undegraded":"2024-01-20T08:31:16.159100+0000","last_fullsized":"2024-01-20T08:31:16.159100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375185+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:31:17.375185+0000","last_peered":"2024-01-20T08:31:17.375185+0000","last_clean":"2024-01-20T08:31:17.375185+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:31:17.375185+0000","last_undegraded":"2024-01-20T08:31:17.375185+0000","last_fullsized":"2024-01-20T08:31:17.375185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.099197+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:31:17.099197+0000","last_peered":"2024-01-20T08:31:17.099197+0000","last_clean":"2024-01-20T08:31:17.099197+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:31:17.099197+0000","last_undegraded":"2024-01-20T08:31:17.099197+0000","last_fullsized":"2024-01-20T08:31:17.099197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089772+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:31:16.089772+0000","last_peered":"2024-01-20T08:31:16.089772+0000","last_clean":"2024-01-20T08:31:16.089772+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:31:16.089772+0000","last_undegraded":"2024-01-20T08:31:16.089772+0000","last_fullsized":"2024-01-20T08:31:16.089772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136303+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:31:17.136303+0000","last_peered":"2024-01-20T08:31:17.136303+0000","last_clean":"2024-01-20T08:31:17.136303+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:31:17.136303+0000","last_undegraded":"2024-01-20T08:31:17.136303+0000","last_fullsized":"2024-01-20T08:31:17.136303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374482+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:31:17.374482+0000","last_peered":"2024-01-20T08:31:17.374482+0000","last_clean":"2024-01-20T08:31:17.374482+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:31:17.374482+0000","last_undegraded":"2024-01-20T08:31:17.374482+0000","last_fullsized":"2024-01-20T08:31:17.374482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147617+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:31:16.147617+0000","last_peered":"2024-01-20T08:31:16.147617+0000","last_clean":"2024-01-20T08:31:16.147617+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:31:16.147617+0000","last_undegraded":"2024-01-20T08:31:16.147617+0000","last_fullsized":"2024-01-20T08:31:16.147617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999557+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:31:37.999557+0000","last_peered":"2024-01-20T08:31:37.999557+0000","last_clean":"2024-01-20T08:31:37.999557+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:31:37.999557+0000","last_undegraded":"2024-01-20T08:31:37.999557+0000","last_fullsized":"2024-01-20T08:31:37.999557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170329+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:31:16.170329+0000","last_peered":"2024-01-20T08:31:16.170329+0000","last_clean":"2024-01-20T08:31:16.170329+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:31:16.170329+0000","last_undegraded":"2024-01-20T08:31:16.170329+0000","last_fullsized":"2024-01-20T08:31:16.170329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137211+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:31:17.137211+0000","last_peered":"2024-01-20T08:31:17.137211+0000","last_clean":"2024-01-20T08:31:17.137211+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:31:17.137211+0000","last_undegraded":"2024-01-20T08:31:17.137211+0000","last_fullsized":"2024-01-20T08:31:17.137211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152169+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:31:16.152169+0000","last_peered":"2024-01-20T08:31:16.152169+0000","last_clean":"2024-01-20T08:31:16.152169+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:31:16.152169+0000","last_undegraded":"2024-01-20T08:31:16.152169+0000","last_fullsized":"2024-01-20T08:31:16.152169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090660+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:31:16.090660+0000","last_peered":"2024-01-20T08:31:16.090660+0000","last_clean":"2024-01-20T08:31:16.090660+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:31:16.090660+0000","last_undegraded":"2024-01-20T08:31:16.090660+0000","last_fullsized":"2024-01-20T08:31:16.090660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374188+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:31:17.374188+0000","last_peered":"2024-01-20T08:31:17.374188+0000","last_clean":"2024-01-20T08:31:17.374188+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:31:17.374188+0000","last_undegraded":"2024-01-20T08:31:17.374188+0000","last_fullsized":"2024-01-20T08:31:17.374188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087660+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:31:16.087660+0000","last_peered":"2024-01-20T08:31:16.087660+0000","last_clean":"2024-01-20T08:31:16.087660+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:31:16.087660+0000","last_undegraded":"2024-01-20T08:31:16.087660+0000","last_fullsized":"2024-01-20T08:31:16.087660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159631+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:31:16.159631+0000","last_peered":"2024-01-20T08:31:16.159631+0000","last_clean":"2024-01-20T08:31:16.159631+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:31:16.159631+0000","last_undegraded":"2024-01-20T08:31:16.159631+0000","last_fullsized":"2024-01-20T08:31:16.159631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375070+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:31:17.375070+0000","last_peered":"2024-01-20T08:31:17.375070+0000","last_clean":"2024-01-20T08:31:17.375070+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:31:17.375070+0000","last_undegraded":"2024-01-20T08:31:17.375070+0000","last_fullsized":"2024-01-20T08:31:17.375070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087747+0000","last_change":"2024-01-20T08:31:10.281433+0000","last_active":"2024-01-20T08:31:16.087747+0000","last_peered":"2024-01-20T08:31:16.087747+0000","last_clean":"2024-01-20T08:31:16.087747+0000","last_became_active":"2024-01-20T08:31:10.281180+0000","last_became_peered":"2024-01-20T08:31:10.281180+0000","last_unstale":"2024-01-20T08:31:16.087747+0000","last_undegraded":"2024-01-20T08:31:16.087747+0000","last_fullsized":"2024-01-20T08:31:16.087747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:04.030668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147775+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:31:16.147775+0000","last_peered":"2024-01-20T08:31:16.147775+0000","last_clean":"2024-01-20T08:31:16.147775+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:31:16.147775+0000","last_undegraded":"2024-01-20T08:31:16.147775+0000","last_fullsized":"2024-01-20T08:31:16.147775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136114+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:31:17.136114+0000","last_peered":"2024-01-20T08:31:17.136114+0000","last_clean":"2024-01-20T08:31:17.136114+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:31:17.136114+0000","last_undegraded":"2024-01-20T08:31:17.136114+0000","last_fullsized":"2024-01-20T08:31:17.136114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.172904+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:31:16.172904+0000","last_peered":"2024-01-20T08:31:16.172904+0000","last_clean":"2024-01-20T08:31:16.172904+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:31:16.172904+0000","last_undegraded":"2024-01-20T08:31:16.172904+0000","last_fullsized":"2024-01-20T08:31:16.172904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087863+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:31:16.087863+0000","last_peered":"2024-01-20T08:31:16.087863+0000","last_clean":"2024-01-20T08:31:16.087863+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:31:16.087863+0000","last_undegraded":"2024-01-20T08:31:16.087863+0000","last_fullsized":"2024-01-20T08:31:16.087863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374106+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:31:17.374106+0000","last_peered":"2024-01-20T08:31:17.374106+0000","last_clean":"2024-01-20T08:31:17.374106+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:31:17.374106+0000","last_undegraded":"2024-01-20T08:31:17.374106+0000","last_fullsized":"2024-01-20T08:31:17.374106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151503+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:31:16.151503+0000","last_peered":"2024-01-20T08:31:16.151503+0000","last_clean":"2024-01-20T08:31:16.151503+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:31:16.151503+0000","last_undegraded":"2024-01-20T08:31:16.151503+0000","last_fullsized":"2024-01-20T08:31:16.151503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147470+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:31:16.147470+0000","last_peered":"2024-01-20T08:31:16.147470+0000","last_clean":"2024-01-20T08:31:16.147470+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:31:16.147470+0000","last_undegraded":"2024-01-20T08:31:16.147470+0000","last_fullsized":"2024-01-20T08:31:16.147470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374628+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:31:17.374628+0000","last_peered":"2024-01-20T08:31:17.374628+0000","last_clean":"2024-01-20T08:31:17.374628+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:31:17.374628+0000","last_undegraded":"2024-01-20T08:31:17.374628+0000","last_fullsized":"2024-01-20T08:31:17.374628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170167+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:31:16.170167+0000","last_peered":"2024-01-20T08:31:16.170167+0000","last_clean":"2024-01-20T08:31:16.170167+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:31:16.170167+0000","last_undegraded":"2024-01-20T08:31:16.170167+0000","last_fullsized":"2024-01-20T08:31:16.170167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087291+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:31:16.087291+0000","last_peered":"2024-01-20T08:31:16.087291+0000","last_clean":"2024-01-20T08:31:16.087291+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:31:16.087291+0000","last_undegraded":"2024-01-20T08:31:16.087291+0000","last_fullsized":"2024-01-20T08:31:16.087291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374579+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:31:17.374579+0000","last_peered":"2024-01-20T08:31:17.374579+0000","last_clean":"2024-01-20T08:31:17.374579+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:31:17.374579+0000","last_undegraded":"2024-01-20T08:31:17.374579+0000","last_fullsized":"2024-01-20T08:31:17.374579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151834+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:31:16.151834+0000","last_peered":"2024-01-20T08:31:16.151834+0000","last_clean":"2024-01-20T08:31:16.151834+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:31:16.151834+0000","last_undegraded":"2024-01-20T08:31:16.151834+0000","last_fullsized":"2024-01-20T08:31:16.151834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999456+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:31:37.999456+0000","last_peered":"2024-01-20T08:31:37.999456+0000","last_clean":"2024-01-20T08:31:37.999456+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:31:37.999456+0000","last_undegraded":"2024-01-20T08:31:37.999456+0000","last_fullsized":"2024-01-20T08:31:37.999456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098815+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:31:17.098815+0000","last_peered":"2024-01-20T08:31:17.098815+0000","last_clean":"2024-01-20T08:31:17.098815+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:31:17.098815+0000","last_undegraded":"2024-01-20T08:31:17.098815+0000","last_fullsized":"2024-01-20T08:31:17.098815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089976+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:31:16.089976+0000","last_peered":"2024-01-20T08:31:16.089976+0000","last_clean":"2024-01-20T08:31:16.089976+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:31:16.089976+0000","last_undegraded":"2024-01-20T08:31:16.089976+0000","last_fullsized":"2024-01-20T08:31:16.089976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.911357+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:31:37.911357+0000","last_peered":"2024-01-20T08:31:37.911357+0000","last_clean":"2024-01-20T08:31:37.911357+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:31:37.911357+0000","last_undegraded":"2024-01-20T08:31:37.911357+0000","last_fullsized":"2024-01-20T08:31:37.911357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136935+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:31:17.136935+0000","last_peered":"2024-01-20T08:31:17.136935+0000","last_clean":"2024-01-20T08:31:17.136935+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:31:17.136935+0000","last_undegraded":"2024-01-20T08:31:17.136935+0000","last_fullsized":"2024-01-20T08:31:17.136935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087400+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:31:16.087400+0000","last_peered":"2024-01-20T08:31:16.087400+0000","last_clean":"2024-01-20T08:31:16.087400+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:31:16.087400+0000","last_undegraded":"2024-01-20T08:31:16.087400+0000","last_fullsized":"2024-01-20T08:31:16.087400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:32.758676+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:31:32.758676+0000","last_peered":"2024-01-20T08:31:32.758676+0000","last_clean":"2024-01-20T08:31:32.758676+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:31:32.758676+0000","last_undegraded":"2024-01-20T08:31:32.758676+0000","last_fullsized":"2024-01-20T08:31:32.758676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373932+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:31:17.373932+0000","last_peered":"2024-01-20T08:31:17.373932+0000","last_clean":"2024-01-20T08:31:17.373932+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:31:17.373932+0000","last_undegraded":"2024-01-20T08:31:17.373932+0000","last_fullsized":"2024-01-20T08:31:17.373932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151948+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:31:16.151948+0000","last_peered":"2024-01-20T08:31:16.151948+0000","last_clean":"2024-01-20T08:31:16.151948+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:31:16.151948+0000","last_undegraded":"2024-01-20T08:31:16.151948+0000","last_fullsized":"2024-01-20T08:31:16.151948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375435+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:31:17.375435+0000","last_peered":"2024-01-20T08:31:17.375435+0000","last_clean":"2024-01-20T08:31:17.375435+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:31:17.375435+0000","last_undegraded":"2024-01-20T08:31:17.375435+0000","last_fullsized":"2024-01-20T08:31:17.375435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374361+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:31:17.374361+0000","last_peered":"2024-01-20T08:31:17.374361+0000","last_clean":"2024-01-20T08:31:17.374361+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:31:17.374361+0000","last_undegraded":"2024-01-20T08:31:17.374361+0000","last_fullsized":"2024-01-20T08:31:17.374361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159818+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:31:16.159818+0000","last_peered":"2024-01-20T08:31:16.159818+0000","last_clean":"2024-01-20T08:31:16.159818+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:31:16.159818+0000","last_undegraded":"2024-01-20T08:31:16.159818+0000","last_fullsized":"2024-01-20T08:31:16.159818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087737+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:31:16.087737+0000","last_peered":"2024-01-20T08:31:16.087737+0000","last_clean":"2024-01-20T08:31:16.087737+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:31:16.087737+0000","last_undegraded":"2024-01-20T08:31:16.087737+0000","last_fullsized":"2024-01-20T08:31:16.087737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373547+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:31:17.373547+0000","last_peered":"2024-01-20T08:31:17.373547+0000","last_clean":"2024-01-20T08:31:17.373547+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:31:17.373547+0000","last_undegraded":"2024-01-20T08:31:17.373547+0000","last_fullsized":"2024-01-20T08:31:17.373547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.172828+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:31:16.172828+0000","last_peered":"2024-01-20T08:31:16.172828+0000","last_clean":"2024-01-20T08:31:16.172828+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:31:16.172828+0000","last_undegraded":"2024-01-20T08:31:16.172828+0000","last_fullsized":"2024-01-20T08:31:16.172828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.173256+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:31:16.173256+0000","last_peered":"2024-01-20T08:31:16.173256+0000","last_clean":"2024-01-20T08:31:16.173256+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:31:16.173256+0000","last_undegraded":"2024-01-20T08:31:16.173256+0000","last_fullsized":"2024-01-20T08:31:16.173256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374118+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:31:17.374118+0000","last_peered":"2024-01-20T08:31:17.374118+0000","last_clean":"2024-01-20T08:31:17.374118+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:31:17.374118+0000","last_undegraded":"2024-01-20T08:31:17.374118+0000","last_fullsized":"2024-01-20T08:31:17.374118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374650+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:31:17.374650+0000","last_peered":"2024-01-20T08:31:17.374650+0000","last_clean":"2024-01-20T08:31:17.374650+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:31:17.374650+0000","last_undegraded":"2024-01-20T08:31:17.374650+0000","last_fullsized":"2024-01-20T08:31:17.374650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147364+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:31:16.147364+0000","last_peered":"2024-01-20T08:31:16.147364+0000","last_clean":"2024-01-20T08:31:16.147364+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:31:16.147364+0000","last_undegraded":"2024-01-20T08:31:16.147364+0000","last_fullsized":"2024-01-20T08:31:16.147364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136297+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:31:17.136297+0000","last_peered":"2024-01-20T08:31:17.136297+0000","last_clean":"2024-01-20T08:31:17.136297+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:31:17.136297+0000","last_undegraded":"2024-01-20T08:31:17.136297+0000","last_fullsized":"2024-01-20T08:31:17.136297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136696+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:31:17.136696+0000","last_peered":"2024-01-20T08:31:17.136696+0000","last_clean":"2024-01-20T08:31:17.136696+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:31:17.136696+0000","last_undegraded":"2024-01-20T08:31:17.136696+0000","last_fullsized":"2024-01-20T08:31:17.136696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159237+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:31:16.159237+0000","last_peered":"2024-01-20T08:31:16.159237+0000","last_clean":"2024-01-20T08:31:16.159237+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:31:16.159237+0000","last_undegraded":"2024-01-20T08:31:16.159237+0000","last_fullsized":"2024-01-20T08:31:16.159237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089892+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:31:16.089892+0000","last_peered":"2024-01-20T08:31:16.089892+0000","last_clean":"2024-01-20T08:31:16.089892+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:31:16.089892+0000","last_undegraded":"2024-01-20T08:31:16.089892+0000","last_fullsized":"2024-01-20T08:31:16.089892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151952+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:31:16.151952+0000","last_peered":"2024-01-20T08:31:16.151952+0000","last_clean":"2024-01-20T08:31:16.151952+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:31:16.151952+0000","last_undegraded":"2024-01-20T08:31:16.151952+0000","last_fullsized":"2024-01-20T08:31:16.151952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147429+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:31:16.147429+0000","last_peered":"2024-01-20T08:31:16.147429+0000","last_clean":"2024-01-20T08:31:16.147429+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:31:16.147429+0000","last_undegraded":"2024-01-20T08:31:16.147429+0000","last_fullsized":"2024-01-20T08:31:16.147429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375382+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:31:17.375382+0000","last_peered":"2024-01-20T08:31:17.375382+0000","last_clean":"2024-01-20T08:31:17.375382+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:31:17.375382+0000","last_undegraded":"2024-01-20T08:31:17.375382+0000","last_fullsized":"2024-01-20T08:31:17.375382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159772+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:31:16.159772+0000","last_peered":"2024-01-20T08:31:16.159772+0000","last_clean":"2024-01-20T08:31:16.159772+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:31:16.159772+0000","last_undegraded":"2024-01-20T08:31:16.159772+0000","last_fullsized":"2024-01-20T08:31:16.159772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169886+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:31:16.169886+0000","last_peered":"2024-01-20T08:31:16.169886+0000","last_clean":"2024-01-20T08:31:16.169886+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:31:16.169886+0000","last_undegraded":"2024-01-20T08:31:16.169886+0000","last_fullsized":"2024-01-20T08:31:16.169886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375349+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:31:17.375349+0000","last_peered":"2024-01-20T08:31:17.375349+0000","last_clean":"2024-01-20T08:31:17.375349+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:31:17.375349+0000","last_undegraded":"2024-01-20T08:31:17.375349+0000","last_fullsized":"2024-01-20T08:31:17.375349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170884+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:31:16.170884+0000","last_peered":"2024-01-20T08:31:16.170884+0000","last_clean":"2024-01-20T08:31:16.170884+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:31:16.170884+0000","last_undegraded":"2024-01-20T08:31:16.170884+0000","last_fullsized":"2024-01-20T08:31:16.170884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152039+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:31:16.152039+0000","last_peered":"2024-01-20T08:31:16.152039+0000","last_clean":"2024-01-20T08:31:16.152039+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:31:16.152039+0000","last_undegraded":"2024-01-20T08:31:16.152039+0000","last_fullsized":"2024-01-20T08:31:16.152039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151833+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:31:16.151833+0000","last_peered":"2024-01-20T08:31:16.151833+0000","last_clean":"2024-01-20T08:31:16.151833+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:31:16.151833+0000","last_undegraded":"2024-01-20T08:31:16.151833+0000","last_fullsized":"2024-01-20T08:31:16.151833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159743+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:31:16.159743+0000","last_peered":"2024-01-20T08:31:16.159743+0000","last_clean":"2024-01-20T08:31:16.159743+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:31:16.159743+0000","last_undegraded":"2024-01-20T08:31:16.159743+0000","last_fullsized":"2024-01-20T08:31:16.159743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090434+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:31:16.090434+0000","last_peered":"2024-01-20T08:31:16.090434+0000","last_clean":"2024-01-20T08:31:16.090434+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:31:16.090434+0000","last_undegraded":"2024-01-20T08:31:16.090434+0000","last_fullsized":"2024-01-20T08:31:16.090434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090234+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:31:16.090234+0000","last_peered":"2024-01-20T08:31:16.090234+0000","last_clean":"2024-01-20T08:31:16.090234+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:31:16.090234+0000","last_undegraded":"2024-01-20T08:31:16.090234+0000","last_fullsized":"2024-01-20T08:31:16.090234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.172746+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:31:16.172746+0000","last_peered":"2024-01-20T08:31:16.172746+0000","last_clean":"2024-01-20T08:31:16.172746+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:31:16.172746+0000","last_undegraded":"2024-01-20T08:31:16.172746+0000","last_fullsized":"2024-01-20T08:31:16.172746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159653+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:31:16.159653+0000","last_peered":"2024-01-20T08:31:16.159653+0000","last_clean":"2024-01-20T08:31:16.159653+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:31:16.159653+0000","last_undegraded":"2024-01-20T08:31:16.159653+0000","last_fullsized":"2024-01-20T08:31:16.159653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151666+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:31:16.151666+0000","last_peered":"2024-01-20T08:31:16.151666+0000","last_clean":"2024-01-20T08:31:16.151666+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:31:16.151666+0000","last_undegraded":"2024-01-20T08:31:16.151666+0000","last_fullsized":"2024-01-20T08:31:16.151666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159043+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:31:16.159043+0000","last_peered":"2024-01-20T08:31:16.159043+0000","last_clean":"2024-01-20T08:31:16.159043+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:31:16.159043+0000","last_undegraded":"2024-01-20T08:31:16.159043+0000","last_fullsized":"2024-01-20T08:31:16.159043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169805+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:31:16.169805+0000","last_peered":"2024-01-20T08:31:16.169805+0000","last_clean":"2024-01-20T08:31:16.169805+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:31:16.169805+0000","last_undegraded":"2024-01-20T08:31:16.169805+0000","last_fullsized":"2024-01-20T08:31:16.169805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147347+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:31:16.147347+0000","last_peered":"2024-01-20T08:31:16.147347+0000","last_clean":"2024-01-20T08:31:16.147347+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:31:16.147347+0000","last_undegraded":"2024-01-20T08:31:16.147347+0000","last_fullsized":"2024-01-20T08:31:16.147347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087586+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:31:16.087586+0000","last_peered":"2024-01-20T08:31:16.087586+0000","last_clean":"2024-01-20T08:31:16.087586+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:31:16.087586+0000","last_undegraded":"2024-01-20T08:31:16.087586+0000","last_fullsized":"2024-01-20T08:31:16.087586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152300+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:31:16.152300+0000","last_peered":"2024-01-20T08:31:16.152300+0000","last_clean":"2024-01-20T08:31:16.152300+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:31:16.152300+0000","last_undegraded":"2024-01-20T08:31:16.152300+0000","last_fullsized":"2024-01-20T08:31:16.152300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152657+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:31:16.152657+0000","last_peered":"2024-01-20T08:31:16.152657+0000","last_clean":"2024-01-20T08:31:16.152657+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:31:16.152657+0000","last_undegraded":"2024-01-20T08:31:16.152657+0000","last_fullsized":"2024-01-20T08:31:16.152657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152271+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:31:16.152271+0000","last_peered":"2024-01-20T08:31:16.152271+0000","last_clean":"2024-01-20T08:31:16.152271+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:31:16.152271+0000","last_undegraded":"2024-01-20T08:31:16.152271+0000","last_fullsized":"2024-01-20T08:31:16.152271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151666+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:31:16.151666+0000","last_peered":"2024-01-20T08:31:16.151666+0000","last_clean":"2024-01-20T08:31:16.151666+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:31:16.151666+0000","last_undegraded":"2024-01-20T08:31:16.151666+0000","last_fullsized":"2024-01-20T08:31:16.151666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.158963+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:31:16.158963+0000","last_peered":"2024-01-20T08:31:16.158963+0000","last_clean":"2024-01-20T08:31:16.158963+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:31:16.158963+0000","last_undegraded":"2024-01-20T08:31:16.158963+0000","last_fullsized":"2024-01-20T08:31:16.158963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089892+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:31:16.089892+0000","last_peered":"2024-01-20T08:31:16.089892+0000","last_clean":"2024-01-20T08:31:16.089892+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:31:16.089892+0000","last_undegraded":"2024-01-20T08:31:16.089892+0000","last_fullsized":"2024-01-20T08:31:16.089892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090330+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:31:16.090330+0000","last_peered":"2024-01-20T08:31:16.090330+0000","last_clean":"2024-01-20T08:31:16.090330+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:31:16.090330+0000","last_undegraded":"2024-01-20T08:31:16.090330+0000","last_fullsized":"2024-01-20T08:31:16.090330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087503+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:31:16.087503+0000","last_peered":"2024-01-20T08:31:16.087503+0000","last_clean":"2024-01-20T08:31:16.087503+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:31:16.087503+0000","last_undegraded":"2024-01-20T08:31:16.087503+0000","last_fullsized":"2024-01-20T08:31:16.087503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087327+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:31:16.087327+0000","last_peered":"2024-01-20T08:31:16.087327+0000","last_clean":"2024-01-20T08:31:16.087327+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:31:16.087327+0000","last_undegraded":"2024-01-20T08:31:16.087327+0000","last_fullsized":"2024-01-20T08:31:16.087327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087526+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:31:16.087526+0000","last_peered":"2024-01-20T08:31:16.087526+0000","last_clean":"2024-01-20T08:31:16.087526+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:31:16.087526+0000","last_undegraded":"2024-01-20T08:31:16.087526+0000","last_fullsized":"2024-01-20T08:31:16.087526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.173025+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:31:16.173025+0000","last_peered":"2024-01-20T08:31:16.173025+0000","last_clean":"2024-01-20T08:31:16.173025+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:31:16.173025+0000","last_undegraded":"2024-01-20T08:31:16.173025+0000","last_fullsized":"2024-01-20T08:31:16.173025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159022+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:31:16.159022+0000","last_peered":"2024-01-20T08:31:16.159022+0000","last_clean":"2024-01-20T08:31:16.159022+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:31:16.159022+0000","last_undegraded":"2024-01-20T08:31:16.159022+0000","last_fullsized":"2024-01-20T08:31:16.159022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087198+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:31:16.087198+0000","last_peered":"2024-01-20T08:31:16.087198+0000","last_clean":"2024-01-20T08:31:16.087198+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:31:16.087198+0000","last_undegraded":"2024-01-20T08:31:16.087198+0000","last_fullsized":"2024-01-20T08:31:16.087198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170339+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:31:16.170339+0000","last_peered":"2024-01-20T08:31:16.170339+0000","last_clean":"2024-01-20T08:31:16.170339+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:31:16.170339+0000","last_undegraded":"2024-01-20T08:31:16.170339+0000","last_fullsized":"2024-01-20T08:31:16.170339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087386+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:31:16.087386+0000","last_peered":"2024-01-20T08:31:16.087386+0000","last_clean":"2024-01-20T08:31:16.087386+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:31:16.087386+0000","last_undegraded":"2024-01-20T08:31:16.087386+0000","last_fullsized":"2024-01-20T08:31:16.087386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169135+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:31:16.169135+0000","last_peered":"2024-01-20T08:31:16.169135+0000","last_clean":"2024-01-20T08:31:16.169135+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:31:16.169135+0000","last_undegraded":"2024-01-20T08:31:16.169135+0000","last_fullsized":"2024-01-20T08:31:16.169135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159545+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:31:16.159545+0000","last_peered":"2024-01-20T08:31:16.159545+0000","last_clean":"2024-01-20T08:31:16.159545+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:31:16.159545+0000","last_undegraded":"2024-01-20T08:31:16.159545+0000","last_fullsized":"2024-01-20T08:31:16.159545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159745+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:31:16.159745+0000","last_peered":"2024-01-20T08:31:16.159745+0000","last_clean":"2024-01-20T08:31:16.159745+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:31:16.159745+0000","last_undegraded":"2024-01-20T08:31:16.159745+0000","last_fullsized":"2024-01-20T08:31:16.159745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087758+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:31:16.087758+0000","last_peered":"2024-01-20T08:31:16.087758+0000","last_clean":"2024-01-20T08:31:16.087758+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:31:16.087758+0000","last_undegraded":"2024-01-20T08:31:16.087758+0000","last_fullsized":"2024-01-20T08:31:16.087758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5243,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":9882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59,"ondisk_log_size":59,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":103963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":694423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":694401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":104567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:31:40.931 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:31:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.062+0000 7f625ff89640 1 Processor -- start 2024-01-20T08:31:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.062+0000 7f625ff89640 1 -- start start 2024-01-20T08:31:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.063+0000 7f625ff89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6258151620 0x7f625815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.063+0000 7f625ff89640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f62580c4a20 0x7f62580cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.063+0000 7f625ff89640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625815b8e0 con 0x7f62580c8450 2024-01-20T08:31:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.063+0000 7f625ff89640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625806f420 con 0x7f6258151620 2024-01-20T08:31:41.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.063+0000 7f625ff89640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625806f700 con 0x7f62580c4a20 2024-01-20T08:31:41.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.064+0000 7f625dd25640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6258151620 0x7f625815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.064+0000 7f625d524640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f62580c8450 0x7f62580c57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38944/0 (socket says 172.21.15.87:38944) 2024-01-20T08:31:41.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.064+0000 7f625dd25640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6258151620 0x7f625815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37632/0 (socket says 172.21.15.87:37632) 2024-01-20T08:31:41.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.064+0000 7f625d524640 1 -- 172.21.15.87:0/531403757 learned_addr learned my addr 172.21.15.87:0/531403757 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:41.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.064+0000 7f625e526640 1 --2- 172.21.15.87:0/531403757 >> v2:172.21.15.116:3300/0 conn(0x7f62580c4a20 0x7f62580cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.066+0000 7f625dd25640 1 -- 172.21.15.87:0/531403757 >> v2:172.21.15.116:3300/0 conn(0x7f62580c4a20 msgr2=0x7f62580cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.066+0000 7f625dd25640 1 --2- 172.21.15.87:0/531403757 >> v2:172.21.15.116:3300/0 conn(0x7f62580c4a20 0x7f62580cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.066+0000 7f625dd25640 1 -- 172.21.15.87:0/531403757 >> v1:172.21.15.116:6789/0 conn(0x7f62580c8450 legacy=0x7f62580c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.066+0000 7f625dd25640 1 -- 172.21.15.87:0/531403757 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580c7970 con 0x7f6258151620 2024-01-20T08:31:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.068+0000 7f625dd25640 1 --2- 172.21.15.87:0/531403757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6258151620 0x7f625815b3a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f625806ce70 tx=0x7f625402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3b546e2f6063b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:41.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.072+0000 7f625cd23640 1 -- 172.21.15.87:0/531403757 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625402cc80 con 0x7f6258151620 2024-01-20T08:31:41.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.072+0000 7f625cd23640 1 -- 172.21.15.87:0/531403757 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625402cde0 con 0x7f6258151620 2024-01-20T08:31:41.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.072+0000 7f625cd23640 1 -- 172.21.15.87:0/531403757 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254034660 con 0x7f6258151620 2024-01-20T08:31:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 -- 172.21.15.87:0/531403757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6258151620 msgr2=0x7f625815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 --2- 172.21.15.87:0/531403757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6258151620 0x7f625815b3a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f625806ce70 tx=0x7f625402bed0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 -- 172.21.15.87:0/531403757 shutdown_connections 2024-01-20T08:31:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 --2- 172.21.15.87:0/531403757 >> v2:172.21.15.116:3300/0 conn(0x7f62580c4a20 0x7f62580cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 --2- 172.21.15.87:0/531403757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6258151620 0x7f625815b3a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 -- 172.21.15.87:0/531403757 >> 172.21.15.87:0/531403757 conn(0x7f625806d510 msgr2=0x7f62580663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 -- 172.21.15.87:0/531403757 shutdown_connections 2024-01-20T08:31:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.073+0000 7f625ff89640 1 -- 172.21.15.87:0/531403757 wait complete. 2024-01-20T08:31:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.074+0000 7f625ff89640 1 Processor -- start 2024-01-20T08:31:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.074+0000 7f625ff89640 1 -- start start 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625ff89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62580c4a20 0x7f62580c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625ff89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 0x7f62581ece60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625ff89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6258151620 0x7f62581494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625ff89640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62581e4630 con 0x7f62580c4a20 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625ff89640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62581f1230 con 0x7f62580c8450 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625ff89640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580ca860 con 0x7f6258151620 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625dd25640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62580c4a20 0x7f62580c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625d524640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 0x7f62581ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.075+0000 7f625e526640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6258151620 0x7f62581494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.076+0000 7f625d524640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 0x7f62581ece60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50064/0 (socket says 172.21.15.87:50064) 2024-01-20T08:31:41.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.076+0000 7f625d524640 1 -- 172.21.15.87:0/2578627407 learned_addr learned my addr 172.21.15.87:0/2578627407 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:41.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.079+0000 7f625d524640 1 -- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6258151620 msgr2=0x7f62581494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.079+0000 7f625d524640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6258151620 0x7f62581494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.079+0000 7f625d524640 1 -- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62580c4a20 msgr2=0x7f62580c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.079+0000 7f625d524640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62580c4a20 0x7f62580c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.079+0000 7f625d524640 1 -- 172.21.15.87:0/2578627407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254009a70 con 0x7f62580c8450 2024-01-20T08:31:41.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.081+0000 7f625d524640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 0x7f62581ece60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f624800c400 tx=0x7f624800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:41.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.085+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248005ce0 con 0x7f62580c8450 2024-01-20T08:31:41.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.085+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248005e40 con 0x7f62580c8450 2024-01-20T08:31:41.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.085+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480072d0 con 0x7f62580c8450 2024-01-20T08:31:41.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.085+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62581e4350 con 0x7f62580c8450 2024-01-20T08:31:41.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.085+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62581f0750 con 0x7f62580c8450 2024-01-20T08:31:41.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.086+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62580c7600 con 0x7f62580c8450 2024-01-20T08:31:41.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.328+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f62480074f0 con 0x7f62580c8450 2024-01-20T08:31:41.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.329+0000 7f624e7fc640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f623c06af20 0x7f623c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.329+0000 7f625dd25640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f623c06af20 0x7f623c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.329+0000 7f625dd25640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f623c06af20 0x7f623c06d3e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6254036000 tx=0x7f6254030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:41.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.333+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6248057ca0 con 0x7f62580c8450 2024-01-20T08:31:41.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.344+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6248053fe0 con 0x7f62580c8450 2024-01-20T08:31:41.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.547+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6258066950 con 0x7f623c06af20 2024-01-20T08:31:41.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.560+0000 7f624e7fc640 1 -- 172.21.15.87:0/2578627407 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345253 (secure 0 0 0) 0x7f6258066950 con 0x7f623c06af20 2024-01-20T08:31:41.566 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f623c06af20 msgr2=0x7f623c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f623c06af20 0x7f623c06d3e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6254036000 tx=0x7f6254030000 comp rx=0 tx=0).stop 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 msgr2=0x7f62581ece60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 0x7f62581ece60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f624800c400 tx=0x7f624800c8c0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 shutdown_connections 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f623c06af20 0x7f623c06d3e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6258151620 0x7f62581494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62580c8450 0x7f62581ece60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 --2- 172.21.15.87:0/2578627407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62580c4a20 0x7f62580c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.563+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 >> 172.21.15.87:0/2578627407 conn(0x7f625806d510 msgr2=0x7f625814ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.564+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 shutdown_connections 2024-01-20T08:31:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.564+0000 7f625ff89640 1 -- 172.21.15.87:0/2578627407 wait complete. 2024-01-20T08:31:41.578 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2024-01-20T08:31:40.283865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595611,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":225,"num_write_kb":2443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219456,"kb_used_data":3280,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755264,"statfs":{"total":773094113280,"available":772869390336,"internally_reserved":0,"allocated":3358720,"data_stored":2022322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":745,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002807"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147627+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:31:16.147627+0000","last_peered":"2024-01-20T08:31:16.147627+0000","last_clean":"2024-01-20T08:31:16.147627+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:31:16.147627+0000","last_undegraded":"2024-01-20T08:31:16.147627+0000","last_fullsized":"2024-01-20T08:31:16.147627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159832+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:31:16.159832+0000","last_peered":"2024-01-20T08:31:16.159832+0000","last_clean":"2024-01-20T08:31:16.159832+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:31:16.159832+0000","last_undegraded":"2024-01-20T08:31:16.159832+0000","last_fullsized":"2024-01-20T08:31:16.159832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375104+0000","last_change":"2024-01-20T08:31:16.175747+0000","last_active":"2024-01-20T08:31:17.375104+0000","last_peered":"2024-01-20T08:31:17.375104+0000","last_clean":"2024-01-20T08:31:17.375104+0000","last_became_active":"2024-01-20T08:31:16.174647+0000","last_became_peered":"2024-01-20T08:31:16.174647+0000","last_unstale":"2024-01-20T08:31:17.375104+0000","last_undegraded":"2024-01-20T08:31:17.375104+0000","last_fullsized":"2024-01-20T08:31:17.375104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:26:34.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159915+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:31:16.159915+0000","last_peered":"2024-01-20T08:31:16.159915+0000","last_clean":"2024-01-20T08:31:16.159915+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:31:16.159915+0000","last_undegraded":"2024-01-20T08:31:16.159915+0000","last_fullsized":"2024-01-20T08:31:16.159915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159743+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:31:16.159743+0000","last_peered":"2024-01-20T08:31:16.159743+0000","last_clean":"2024-01-20T08:31:16.159743+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:31:16.159743+0000","last_undegraded":"2024-01-20T08:31:16.159743+0000","last_fullsized":"2024-01-20T08:31:16.159743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373746+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:31:17.373746+0000","last_peered":"2024-01-20T08:31:17.373746+0000","last_clean":"2024-01-20T08:31:17.373746+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:31:17.373746+0000","last_undegraded":"2024-01-20T08:31:17.373746+0000","last_fullsized":"2024-01-20T08:31:17.373746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087496+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:31:16.087496+0000","last_peered":"2024-01-20T08:31:16.087496+0000","last_clean":"2024-01-20T08:31:16.087496+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:31:16.087496+0000","last_undegraded":"2024-01-20T08:31:16.087496+0000","last_fullsized":"2024-01-20T08:31:16.087496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.171846+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:31:16.171846+0000","last_peered":"2024-01-20T08:31:16.171846+0000","last_clean":"2024-01-20T08:31:16.171846+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:31:16.171846+0000","last_undegraded":"2024-01-20T08:31:16.171846+0000","last_fullsized":"2024-01-20T08:31:16.171846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374782+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:31:17.374782+0000","last_peered":"2024-01-20T08:31:17.374782+0000","last_clean":"2024-01-20T08:31:17.374782+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:31:17.374782+0000","last_undegraded":"2024-01-20T08:31:17.374782+0000","last_fullsized":"2024-01-20T08:31:17.374782+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087803+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:31:16.087803+0000","last_peered":"2024-01-20T08:31:16.087803+0000","last_clean":"2024-01-20T08:31:16.087803+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:31:16.087803+0000","last_undegraded":"2024-01-20T08:31:16.087803+0000","last_fullsized":"2024-01-20T08:31:16.087803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152170+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:31:16.152170+0000","last_peered":"2024-01-20T08:31:16.152170+0000","last_clean":"2024-01-20T08:31:16.152170+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:31:16.152170+0000","last_undegraded":"2024-01-20T08:31:16.152170+0000","last_fullsized":"2024-01-20T08:31:16.152170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374153+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:31:17.374153+0000","last_peered":"2024-01-20T08:31:17.374153+0000","last_clean":"2024-01-20T08:31:17.374153+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:31:17.374153+0000","last_undegraded":"2024-01-20T08:31:17.374153+0000","last_fullsized":"2024-01-20T08:31:17.374153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152179+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:31:16.152179+0000","last_peered":"2024-01-20T08:31:16.152179+0000","last_clean":"2024-01-20T08:31:16.152179+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:31:16.152179+0000","last_undegraded":"2024-01-20T08:31:16.152179+0000","last_fullsized":"2024-01-20T08:31:16.152179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999492+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:31:37.999492+0000","last_peered":"2024-01-20T08:31:37.999492+0000","last_clean":"2024-01-20T08:31:37.999492+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:31:37.999492+0000","last_undegraded":"2024-01-20T08:31:37.999492+0000","last_fullsized":"2024-01-20T08:31:37.999492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137443+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:31:17.137443+0000","last_peered":"2024-01-20T08:31:17.137443+0000","last_clean":"2024-01-20T08:31:17.137443+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:31:17.137443+0000","last_undegraded":"2024-01-20T08:31:17.137443+0000","last_fullsized":"2024-01-20T08:31:17.137443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152379+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:31:16.152379+0000","last_peered":"2024-01-20T08:31:16.152379+0000","last_clean":"2024-01-20T08:31:16.152379+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:31:16.152379+0000","last_undegraded":"2024-01-20T08:31:16.152379+0000","last_fullsized":"2024-01-20T08:31:16.152379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090103+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:31:16.090103+0000","last_peered":"2024-01-20T08:31:16.090103+0000","last_clean":"2024-01-20T08:31:16.090103+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:31:16.090103+0000","last_undegraded":"2024-01-20T08:31:16.090103+0000","last_fullsized":"2024-01-20T08:31:16.090103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.188856+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:31:17.188856+0000","last_peered":"2024-01-20T08:31:17.188856+0000","last_clean":"2024-01-20T08:31:17.188856+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:31:17.188856+0000","last_undegraded":"2024-01-20T08:31:17.188856+0000","last_fullsized":"2024-01-20T08:31:17.188856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147252+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:31:16.147252+0000","last_peered":"2024-01-20T08:31:16.147252+0000","last_clean":"2024-01-20T08:31:16.147252+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:31:16.147252+0000","last_undegraded":"2024-01-20T08:31:16.147252+0000","last_fullsized":"2024-01-20T08:31:16.147252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147399+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:31:16.147399+0000","last_peered":"2024-01-20T08:31:16.147399+0000","last_clean":"2024-01-20T08:31:16.147399+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:31:16.147399+0000","last_undegraded":"2024-01-20T08:31:16.147399+0000","last_fullsized":"2024-01-20T08:31:16.147399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374822+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:31:17.374822+0000","last_peered":"2024-01-20T08:31:17.374822+0000","last_clean":"2024-01-20T08:31:17.374822+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:31:17.374822+0000","last_undegraded":"2024-01-20T08:31:17.374822+0000","last_fullsized":"2024-01-20T08:31:17.374822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151575+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:31:16.151575+0000","last_peered":"2024-01-20T08:31:16.151575+0000","last_clean":"2024-01-20T08:31:16.151575+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:31:16.151575+0000","last_undegraded":"2024-01-20T08:31:16.151575+0000","last_fullsized":"2024-01-20T08:31:16.151575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087150+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:31:16.087150+0000","last_peered":"2024-01-20T08:31:16.087150+0000","last_clean":"2024-01-20T08:31:16.087150+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:31:16.087150+0000","last_undegraded":"2024-01-20T08:31:16.087150+0000","last_fullsized":"2024-01-20T08:31:16.087150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374458+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:31:17.374458+0000","last_peered":"2024-01-20T08:31:17.374458+0000","last_clean":"2024-01-20T08:31:17.374458+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:31:17.374458+0000","last_undegraded":"2024-01-20T08:31:17.374458+0000","last_fullsized":"2024-01-20T08:31:17.374458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147691+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:31:16.147691+0000","last_peered":"2024-01-20T08:31:16.147691+0000","last_clean":"2024-01-20T08:31:16.147691+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:31:16.147691+0000","last_undegraded":"2024-01-20T08:31:16.147691+0000","last_fullsized":"2024-01-20T08:31:16.147691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.998883+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:31:37.998883+0000","last_peered":"2024-01-20T08:31:37.998883+0000","last_clean":"2024-01-20T08:31:37.998883+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:31:37.998883+0000","last_undegraded":"2024-01-20T08:31:37.998883+0000","last_fullsized":"2024-01-20T08:31:37.998883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374019+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:31:17.374019+0000","last_peered":"2024-01-20T08:31:17.374019+0000","last_clean":"2024-01-20T08:31:17.374019+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:31:17.374019+0000","last_undegraded":"2024-01-20T08:31:17.374019+0000","last_fullsized":"2024-01-20T08:31:17.374019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147548+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:31:16.147548+0000","last_peered":"2024-01-20T08:31:16.147548+0000","last_clean":"2024-01-20T08:31:16.147548+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:31:16.147548+0000","last_undegraded":"2024-01-20T08:31:16.147548+0000","last_fullsized":"2024-01-20T08:31:16.147548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.293427+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:31:16.293427+0000","last_peered":"2024-01-20T08:31:16.293427+0000","last_clean":"2024-01-20T08:31:16.293427+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:31:16.293427+0000","last_undegraded":"2024-01-20T08:31:16.293427+0000","last_fullsized":"2024-01-20T08:31:16.293427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373815+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:31:17.373815+0000","last_peered":"2024-01-20T08:31:17.373815+0000","last_clean":"2024-01-20T08:31:17.373815+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:31:17.373815+0000","last_undegraded":"2024-01-20T08:31:17.373815+0000","last_fullsized":"2024-01-20T08:31:17.373815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090530+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:31:16.090530+0000","last_peered":"2024-01-20T08:31:16.090530+0000","last_clean":"2024-01-20T08:31:16.090530+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:31:16.090530+0000","last_undegraded":"2024-01-20T08:31:16.090530+0000","last_fullsized":"2024-01-20T08:31:16.090530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137361+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:31:17.137361+0000","last_peered":"2024-01-20T08:31:17.137361+0000","last_clean":"2024-01-20T08:31:17.137361+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:31:17.137361+0000","last_undegraded":"2024-01-20T08:31:17.137361+0000","last_fullsized":"2024-01-20T08:31:17.137361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098675+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:31:17.098675+0000","last_peered":"2024-01-20T08:31:17.098675+0000","last_clean":"2024-01-20T08:31:17.098675+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:31:17.098675+0000","last_undegraded":"2024-01-20T08:31:17.098675+0000","last_fullsized":"2024-01-20T08:31:17.098675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169805+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:31:16.169805+0000","last_peered":"2024-01-20T08:31:16.169805+0000","last_clean":"2024-01-20T08:31:16.169805+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:31:16.169805+0000","last_undegraded":"2024-01-20T08:31:16.169805+0000","last_fullsized":"2024-01-20T08:31:16.169805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159306+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:31:16.159306+0000","last_peered":"2024-01-20T08:31:16.159306+0000","last_clean":"2024-01-20T08:31:16.159306+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:31:16.159306+0000","last_undegraded":"2024-01-20T08:31:16.159306+0000","last_fullsized":"2024-01-20T08:31:16.159306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098613+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:31:17.098613+0000","last_peered":"2024-01-20T08:31:17.098613+0000","last_clean":"2024-01-20T08:31:17.098613+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:31:17.098613+0000","last_undegraded":"2024-01-20T08:31:17.098613+0000","last_fullsized":"2024-01-20T08:31:17.098613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087311+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:31:16.087311+0000","last_peered":"2024-01-20T08:31:16.087311+0000","last_clean":"2024-01-20T08:31:16.087311+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:31:16.087311+0000","last_undegraded":"2024-01-20T08:31:16.087311+0000","last_fullsized":"2024-01-20T08:31:16.087311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152112+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:31:16.152112+0000","last_peered":"2024-01-20T08:31:16.152112+0000","last_clean":"2024-01-20T08:31:16.152112+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:31:16.152112+0000","last_undegraded":"2024-01-20T08:31:16.152112+0000","last_fullsized":"2024-01-20T08:31:16.152112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098960+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:31:17.098960+0000","last_peered":"2024-01-20T08:31:17.098960+0000","last_clean":"2024-01-20T08:31:17.098960+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:31:17.098960+0000","last_undegraded":"2024-01-20T08:31:17.098960+0000","last_fullsized":"2024-01-20T08:31:17.098960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159100+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:31:16.159100+0000","last_peered":"2024-01-20T08:31:16.159100+0000","last_clean":"2024-01-20T08:31:16.159100+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:31:16.159100+0000","last_undegraded":"2024-01-20T08:31:16.159100+0000","last_fullsized":"2024-01-20T08:31:16.159100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375185+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:31:17.375185+0000","last_peered":"2024-01-20T08:31:17.375185+0000","last_clean":"2024-01-20T08:31:17.375185+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:31:17.375185+0000","last_undegraded":"2024-01-20T08:31:17.375185+0000","last_fullsized":"2024-01-20T08:31:17.375185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.099197+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:31:17.099197+0000","last_peered":"2024-01-20T08:31:17.099197+0000","last_clean":"2024-01-20T08:31:17.099197+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:31:17.099197+0000","last_undegraded":"2024-01-20T08:31:17.099197+0000","last_fullsized":"2024-01-20T08:31:17.099197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089772+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:31:16.089772+0000","last_peered":"2024-01-20T08:31:16.089772+0000","last_clean":"2024-01-20T08:31:16.089772+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:31:16.089772+0000","last_undegraded":"2024-01-20T08:31:16.089772+0000","last_fullsized":"2024-01-20T08:31:16.089772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136303+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:31:17.136303+0000","last_peered":"2024-01-20T08:31:17.136303+0000","last_clean":"2024-01-20T08:31:17.136303+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:31:17.136303+0000","last_undegraded":"2024-01-20T08:31:17.136303+0000","last_fullsized":"2024-01-20T08:31:17.136303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374482+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:31:17.374482+0000","last_peered":"2024-01-20T08:31:17.374482+0000","last_clean":"2024-01-20T08:31:17.374482+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:31:17.374482+0000","last_undegraded":"2024-01-20T08:31:17.374482+0000","last_fullsized":"2024-01-20T08:31:17.374482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147617+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:31:16.147617+0000","last_peered":"2024-01-20T08:31:16.147617+0000","last_clean":"2024-01-20T08:31:16.147617+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:31:16.147617+0000","last_undegraded":"2024-01-20T08:31:16.147617+0000","last_fullsized":"2024-01-20T08:31:16.147617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999557+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:31:37.999557+0000","last_peered":"2024-01-20T08:31:37.999557+0000","last_clean":"2024-01-20T08:31:37.999557+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:31:37.999557+0000","last_undegraded":"2024-01-20T08:31:37.999557+0000","last_fullsized":"2024-01-20T08:31:37.999557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170329+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:31:16.170329+0000","last_peered":"2024-01-20T08:31:16.170329+0000","last_clean":"2024-01-20T08:31:16.170329+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:31:16.170329+0000","last_undegraded":"2024-01-20T08:31:16.170329+0000","last_fullsized":"2024-01-20T08:31:16.170329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137211+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:31:17.137211+0000","last_peered":"2024-01-20T08:31:17.137211+0000","last_clean":"2024-01-20T08:31:17.137211+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:31:17.137211+0000","last_undegraded":"2024-01-20T08:31:17.137211+0000","last_fullsized":"2024-01-20T08:31:17.137211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152169+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:31:16.152169+0000","last_peered":"2024-01-20T08:31:16.152169+0000","last_clean":"2024-01-20T08:31:16.152169+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:31:16.152169+0000","last_undegraded":"2024-01-20T08:31:16.152169+0000","last_fullsized":"2024-01-20T08:31:16.152169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090660+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:31:16.090660+0000","last_peered":"2024-01-20T08:31:16.090660+0000","last_clean":"2024-01-20T08:31:16.090660+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:31:16.090660+0000","last_undegraded":"2024-01-20T08:31:16.090660+0000","last_fullsized":"2024-01-20T08:31:16.090660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374188+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:31:17.374188+0000","last_peered":"2024-01-20T08:31:17.374188+0000","last_clean":"2024-01-20T08:31:17.374188+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:31:17.374188+0000","last_undegraded":"2024-01-20T08:31:17.374188+0000","last_fullsized":"2024-01-20T08:31:17.374188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087660+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:31:16.087660+0000","last_peered":"2024-01-20T08:31:16.087660+0000","last_clean":"2024-01-20T08:31:16.087660+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:31:16.087660+0000","last_undegraded":"2024-01-20T08:31:16.087660+0000","last_fullsized":"2024-01-20T08:31:16.087660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159631+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:31:16.159631+0000","last_peered":"2024-01-20T08:31:16.159631+0000","last_clean":"2024-01-20T08:31:16.159631+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:31:16.159631+0000","last_undegraded":"2024-01-20T08:31:16.159631+0000","last_fullsized":"2024-01-20T08:31:16.159631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375070+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:31:17.375070+0000","last_peered":"2024-01-20T08:31:17.375070+0000","last_clean":"2024-01-20T08:31:17.375070+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:31:17.375070+0000","last_undegraded":"2024-01-20T08:31:17.375070+0000","last_fullsized":"2024-01-20T08:31:17.375070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087747+0000","last_change":"2024-01-20T08:31:10.281433+0000","last_active":"2024-01-20T08:31:16.087747+0000","last_peered":"2024-01-20T08:31:16.087747+0000","last_clean":"2024-01-20T08:31:16.087747+0000","last_became_active":"2024-01-20T08:31:10.281180+0000","last_became_peered":"2024-01-20T08:31:10.281180+0000","last_unstale":"2024-01-20T08:31:16.087747+0000","last_undegraded":"2024-01-20T08:31:16.087747+0000","last_fullsized":"2024-01-20T08:31:16.087747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:04.030668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147775+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:31:16.147775+0000","last_peered":"2024-01-20T08:31:16.147775+0000","last_clean":"2024-01-20T08:31:16.147775+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:31:16.147775+0000","last_undegraded":"2024-01-20T08:31:16.147775+0000","last_fullsized":"2024-01-20T08:31:16.147775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136114+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:31:17.136114+0000","last_peered":"2024-01-20T08:31:17.136114+0000","last_clean":"2024-01-20T08:31:17.136114+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:31:17.136114+0000","last_undegraded":"2024-01-20T08:31:17.136114+0000","last_fullsized":"2024-01-20T08:31:17.136114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.172904+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:31:16.172904+0000","last_peered":"2024-01-20T08:31:16.172904+0000","last_clean":"2024-01-20T08:31:16.172904+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:31:16.172904+0000","last_undegraded":"2024-01-20T08:31:16.172904+0000","last_fullsized":"2024-01-20T08:31:16.172904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087863+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:31:16.087863+0000","last_peered":"2024-01-20T08:31:16.087863+0000","last_clean":"2024-01-20T08:31:16.087863+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:31:16.087863+0000","last_undegraded":"2024-01-20T08:31:16.087863+0000","last_fullsized":"2024-01-20T08:31:16.087863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374106+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:31:17.374106+0000","last_peered":"2024-01-20T08:31:17.374106+0000","last_clean":"2024-01-20T08:31:17.374106+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:31:17.374106+0000","last_undegraded":"2024-01-20T08:31:17.374106+0000","last_fullsized":"2024-01-20T08:31:17.374106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151503+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:31:16.151503+0000","last_peered":"2024-01-20T08:31:16.151503+0000","last_clean":"2024-01-20T08:31:16.151503+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:31:16.151503+0000","last_undegraded":"2024-01-20T08:31:16.151503+0000","last_fullsized":"2024-01-20T08:31:16.151503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147470+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:31:16.147470+0000","last_peered":"2024-01-20T08:31:16.147470+0000","last_clean":"2024-01-20T08:31:16.147470+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:31:16.147470+0000","last_undegraded":"2024-01-20T08:31:16.147470+0000","last_fullsized":"2024-01-20T08:31:16.147470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374628+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:31:17.374628+0000","last_peered":"2024-01-20T08:31:17.374628+0000","last_clean":"2024-01-20T08:31:17.374628+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:31:17.374628+0000","last_undegraded":"2024-01-20T08:31:17.374628+0000","last_fullsized":"2024-01-20T08:31:17.374628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170167+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:31:16.170167+0000","last_peered":"2024-01-20T08:31:16.170167+0000","last_clean":"2024-01-20T08:31:16.170167+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:31:16.170167+0000","last_undegraded":"2024-01-20T08:31:16.170167+0000","last_fullsized":"2024-01-20T08:31:16.170167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087291+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:31:16.087291+0000","last_peered":"2024-01-20T08:31:16.087291+0000","last_clean":"2024-01-20T08:31:16.087291+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:31:16.087291+0000","last_undegraded":"2024-01-20T08:31:16.087291+0000","last_fullsized":"2024-01-20T08:31:16.087291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374579+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:31:17.374579+0000","last_peered":"2024-01-20T08:31:17.374579+0000","last_clean":"2024-01-20T08:31:17.374579+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:31:17.374579+0000","last_undegraded":"2024-01-20T08:31:17.374579+0000","last_fullsized":"2024-01-20T08:31:17.374579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151834+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:31:16.151834+0000","last_peered":"2024-01-20T08:31:16.151834+0000","last_clean":"2024-01-20T08:31:16.151834+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:31:16.151834+0000","last_undegraded":"2024-01-20T08:31:16.151834+0000","last_fullsized":"2024-01-20T08:31:16.151834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999456+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:31:37.999456+0000","last_peered":"2024-01-20T08:31:37.999456+0000","last_clean":"2024-01-20T08:31:37.999456+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:31:37.999456+0000","last_undegraded":"2024-01-20T08:31:37.999456+0000","last_fullsized":"2024-01-20T08:31:37.999456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.098815+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:31:17.098815+0000","last_peered":"2024-01-20T08:31:17.098815+0000","last_clean":"2024-01-20T08:31:17.098815+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:31:17.098815+0000","last_undegraded":"2024-01-20T08:31:17.098815+0000","last_fullsized":"2024-01-20T08:31:17.098815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089976+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:31:16.089976+0000","last_peered":"2024-01-20T08:31:16.089976+0000","last_clean":"2024-01-20T08:31:16.089976+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:31:16.089976+0000","last_undegraded":"2024-01-20T08:31:16.089976+0000","last_fullsized":"2024-01-20T08:31:16.089976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.911357+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:31:37.911357+0000","last_peered":"2024-01-20T08:31:37.911357+0000","last_clean":"2024-01-20T08:31:37.911357+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:31:37.911357+0000","last_undegraded":"2024-01-20T08:31:37.911357+0000","last_fullsized":"2024-01-20T08:31:37.911357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136935+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:31:17.136935+0000","last_peered":"2024-01-20T08:31:17.136935+0000","last_clean":"2024-01-20T08:31:17.136935+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:31:17.136935+0000","last_undegraded":"2024-01-20T08:31:17.136935+0000","last_fullsized":"2024-01-20T08:31:17.136935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087400+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:31:16.087400+0000","last_peered":"2024-01-20T08:31:16.087400+0000","last_clean":"2024-01-20T08:31:16.087400+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:31:16.087400+0000","last_undegraded":"2024-01-20T08:31:16.087400+0000","last_fullsized":"2024-01-20T08:31:16.087400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:32.758676+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:31:32.758676+0000","last_peered":"2024-01-20T08:31:32.758676+0000","last_clean":"2024-01-20T08:31:32.758676+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:31:32.758676+0000","last_undegraded":"2024-01-20T08:31:32.758676+0000","last_fullsized":"2024-01-20T08:31:32.758676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373932+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:31:17.373932+0000","last_peered":"2024-01-20T08:31:17.373932+0000","last_clean":"2024-01-20T08:31:17.373932+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:31:17.373932+0000","last_undegraded":"2024-01-20T08:31:17.373932+0000","last_fullsized":"2024-01-20T08:31:17.373932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151948+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:31:16.151948+0000","last_peered":"2024-01-20T08:31:16.151948+0000","last_clean":"2024-01-20T08:31:16.151948+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:31:16.151948+0000","last_undegraded":"2024-01-20T08:31:16.151948+0000","last_fullsized":"2024-01-20T08:31:16.151948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375435+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:31:17.375435+0000","last_peered":"2024-01-20T08:31:17.375435+0000","last_clean":"2024-01-20T08:31:17.375435+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:31:17.375435+0000","last_undegraded":"2024-01-20T08:31:17.375435+0000","last_fullsized":"2024-01-20T08:31:17.375435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374361+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:31:17.374361+0000","last_peered":"2024-01-20T08:31:17.374361+0000","last_clean":"2024-01-20T08:31:17.374361+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:31:17.374361+0000","last_undegraded":"2024-01-20T08:31:17.374361+0000","last_fullsized":"2024-01-20T08:31:17.374361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159818+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:31:16.159818+0000","last_peered":"2024-01-20T08:31:16.159818+0000","last_clean":"2024-01-20T08:31:16.159818+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:31:16.159818+0000","last_undegraded":"2024-01-20T08:31:16.159818+0000","last_fullsized":"2024-01-20T08:31:16.159818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087737+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:31:16.087737+0000","last_peered":"2024-01-20T08:31:16.087737+0000","last_clean":"2024-01-20T08:31:16.087737+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:31:16.087737+0000","last_undegraded":"2024-01-20T08:31:16.087737+0000","last_fullsized":"2024-01-20T08:31:16.087737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.373547+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:31:17.373547+0000","last_peered":"2024-01-20T08:31:17.373547+0000","last_clean":"2024-01-20T08:31:17.373547+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:31:17.373547+0000","last_undegraded":"2024-01-20T08:31:17.373547+0000","last_fullsized":"2024-01-20T08:31:17.373547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.172828+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:31:16.172828+0000","last_peered":"2024-01-20T08:31:16.172828+0000","last_clean":"2024-01-20T08:31:16.172828+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:31:16.172828+0000","last_undegraded":"2024-01-20T08:31:16.172828+0000","last_fullsized":"2024-01-20T08:31:16.172828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.173256+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:31:16.173256+0000","last_peered":"2024-01-20T08:31:16.173256+0000","last_clean":"2024-01-20T08:31:16.173256+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:31:16.173256+0000","last_undegraded":"2024-01-20T08:31:16.173256+0000","last_fullsized":"2024-01-20T08:31:16.173256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374118+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:31:17.374118+0000","last_peered":"2024-01-20T08:31:17.374118+0000","last_clean":"2024-01-20T08:31:17.374118+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:31:17.374118+0000","last_undegraded":"2024-01-20T08:31:17.374118+0000","last_fullsized":"2024-01-20T08:31:17.374118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.374650+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:31:17.374650+0000","last_peered":"2024-01-20T08:31:17.374650+0000","last_clean":"2024-01-20T08:31:17.374650+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:31:17.374650+0000","last_undegraded":"2024-01-20T08:31:17.374650+0000","last_fullsized":"2024-01-20T08:31:17.374650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147364+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:31:16.147364+0000","last_peered":"2024-01-20T08:31:16.147364+0000","last_clean":"2024-01-20T08:31:16.147364+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:31:16.147364+0000","last_undegraded":"2024-01-20T08:31:16.147364+0000","last_fullsized":"2024-01-20T08:31:16.147364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136297+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:31:17.136297+0000","last_peered":"2024-01-20T08:31:17.136297+0000","last_clean":"2024-01-20T08:31:17.136297+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:31:17.136297+0000","last_undegraded":"2024-01-20T08:31:17.136297+0000","last_fullsized":"2024-01-20T08:31:17.136297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136696+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:31:17.136696+0000","last_peered":"2024-01-20T08:31:17.136696+0000","last_clean":"2024-01-20T08:31:17.136696+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:31:17.136696+0000","last_undegraded":"2024-01-20T08:31:17.136696+0000","last_fullsized":"2024-01-20T08:31:17.136696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159237+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:31:16.159237+0000","last_peered":"2024-01-20T08:31:16.159237+0000","last_clean":"2024-01-20T08:31:16.159237+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:31:16.159237+0000","last_undegraded":"2024-01-20T08:31:16.159237+0000","last_fullsized":"2024-01-20T08:31:16.159237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089892+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:31:16.089892+0000","last_peered":"2024-01-20T08:31:16.089892+0000","last_clean":"2024-01-20T08:31:16.089892+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:31:16.089892+0000","last_undegraded":"2024-01-20T08:31:16.089892+0000","last_fullsized":"2024-01-20T08:31:16.089892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151952+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:31:16.151952+0000","last_peered":"2024-01-20T08:31:16.151952+0000","last_clean":"2024-01-20T08:31:16.151952+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:31:16.151952+0000","last_undegraded":"2024-01-20T08:31:16.151952+0000","last_fullsized":"2024-01-20T08:31:16.151952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147429+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:31:16.147429+0000","last_peered":"2024-01-20T08:31:16.147429+0000","last_clean":"2024-01-20T08:31:16.147429+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:31:16.147429+0000","last_undegraded":"2024-01-20T08:31:16.147429+0000","last_fullsized":"2024-01-20T08:31:16.147429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375382+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:31:17.375382+0000","last_peered":"2024-01-20T08:31:17.375382+0000","last_clean":"2024-01-20T08:31:17.375382+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:31:17.375382+0000","last_undegraded":"2024-01-20T08:31:17.375382+0000","last_fullsized":"2024-01-20T08:31:17.375382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159772+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:31:16.159772+0000","last_peered":"2024-01-20T08:31:16.159772+0000","last_clean":"2024-01-20T08:31:16.159772+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:31:16.159772+0000","last_undegraded":"2024-01-20T08:31:16.159772+0000","last_fullsized":"2024-01-20T08:31:16.159772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169886+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:31:16.169886+0000","last_peered":"2024-01-20T08:31:16.169886+0000","last_clean":"2024-01-20T08:31:16.169886+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:31:16.169886+0000","last_undegraded":"2024-01-20T08:31:16.169886+0000","last_fullsized":"2024-01-20T08:31:16.169886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.375349+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:31:17.375349+0000","last_peered":"2024-01-20T08:31:17.375349+0000","last_clean":"2024-01-20T08:31:17.375349+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:31:17.375349+0000","last_undegraded":"2024-01-20T08:31:17.375349+0000","last_fullsized":"2024-01-20T08:31:17.375349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170884+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:31:16.170884+0000","last_peered":"2024-01-20T08:31:16.170884+0000","last_clean":"2024-01-20T08:31:16.170884+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:31:16.170884+0000","last_undegraded":"2024-01-20T08:31:16.170884+0000","last_fullsized":"2024-01-20T08:31:16.170884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152039+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:31:16.152039+0000","last_peered":"2024-01-20T08:31:16.152039+0000","last_clean":"2024-01-20T08:31:16.152039+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:31:16.152039+0000","last_undegraded":"2024-01-20T08:31:16.152039+0000","last_fullsized":"2024-01-20T08:31:16.152039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151833+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:31:16.151833+0000","last_peered":"2024-01-20T08:31:16.151833+0000","last_clean":"2024-01-20T08:31:16.151833+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:31:16.151833+0000","last_undegraded":"2024-01-20T08:31:16.151833+0000","last_fullsized":"2024-01-20T08:31:16.151833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159743+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:31:16.159743+0000","last_peered":"2024-01-20T08:31:16.159743+0000","last_clean":"2024-01-20T08:31:16.159743+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:31:16.159743+0000","last_undegraded":"2024-01-20T08:31:16.159743+0000","last_fullsized":"2024-01-20T08:31:16.159743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090434+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:31:16.090434+0000","last_peered":"2024-01-20T08:31:16.090434+0000","last_clean":"2024-01-20T08:31:16.090434+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:31:16.090434+0000","last_undegraded":"2024-01-20T08:31:16.090434+0000","last_fullsized":"2024-01-20T08:31:16.090434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090234+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:31:16.090234+0000","last_peered":"2024-01-20T08:31:16.090234+0000","last_clean":"2024-01-20T08:31:16.090234+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:31:16.090234+0000","last_undegraded":"2024-01-20T08:31:16.090234+0000","last_fullsized":"2024-01-20T08:31:16.090234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.172746+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:31:16.172746+0000","last_peered":"2024-01-20T08:31:16.172746+0000","last_clean":"2024-01-20T08:31:16.172746+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:31:16.172746+0000","last_undegraded":"2024-01-20T08:31:16.172746+0000","last_fullsized":"2024-01-20T08:31:16.172746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159653+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:31:16.159653+0000","last_peered":"2024-01-20T08:31:16.159653+0000","last_clean":"2024-01-20T08:31:16.159653+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:31:16.159653+0000","last_undegraded":"2024-01-20T08:31:16.159653+0000","last_fullsized":"2024-01-20T08:31:16.159653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151666+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:31:16.151666+0000","last_peered":"2024-01-20T08:31:16.151666+0000","last_clean":"2024-01-20T08:31:16.151666+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:31:16.151666+0000","last_undegraded":"2024-01-20T08:31:16.151666+0000","last_fullsized":"2024-01-20T08:31:16.151666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159043+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:31:16.159043+0000","last_peered":"2024-01-20T08:31:16.159043+0000","last_clean":"2024-01-20T08:31:16.159043+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:31:16.159043+0000","last_undegraded":"2024-01-20T08:31:16.159043+0000","last_fullsized":"2024-01-20T08:31:16.159043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169805+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:31:16.169805+0000","last_peered":"2024-01-20T08:31:16.169805+0000","last_clean":"2024-01-20T08:31:16.169805+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:31:16.169805+0000","last_undegraded":"2024-01-20T08:31:16.169805+0000","last_fullsized":"2024-01-20T08:31:16.169805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.147347+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:31:16.147347+0000","last_peered":"2024-01-20T08:31:16.147347+0000","last_clean":"2024-01-20T08:31:16.147347+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:31:16.147347+0000","last_undegraded":"2024-01-20T08:31:16.147347+0000","last_fullsized":"2024-01-20T08:31:16.147347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087586+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:31:16.087586+0000","last_peered":"2024-01-20T08:31:16.087586+0000","last_clean":"2024-01-20T08:31:16.087586+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:31:16.087586+0000","last_undegraded":"2024-01-20T08:31:16.087586+0000","last_fullsized":"2024-01-20T08:31:16.087586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152300+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:31:16.152300+0000","last_peered":"2024-01-20T08:31:16.152300+0000","last_clean":"2024-01-20T08:31:16.152300+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:31:16.152300+0000","last_undegraded":"2024-01-20T08:31:16.152300+0000","last_fullsized":"2024-01-20T08:31:16.152300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152657+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:31:16.152657+0000","last_peered":"2024-01-20T08:31:16.152657+0000","last_clean":"2024-01-20T08:31:16.152657+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:31:16.152657+0000","last_undegraded":"2024-01-20T08:31:16.152657+0000","last_fullsized":"2024-01-20T08:31:16.152657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.152271+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:31:16.152271+0000","last_peered":"2024-01-20T08:31:16.152271+0000","last_clean":"2024-01-20T08:31:16.152271+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:31:16.152271+0000","last_undegraded":"2024-01-20T08:31:16.152271+0000","last_fullsized":"2024-01-20T08:31:16.152271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.151666+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:31:16.151666+0000","last_peered":"2024-01-20T08:31:16.151666+0000","last_clean":"2024-01-20T08:31:16.151666+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:31:16.151666+0000","last_undegraded":"2024-01-20T08:31:16.151666+0000","last_fullsized":"2024-01-20T08:31:16.151666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.158963+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:31:16.158963+0000","last_peered":"2024-01-20T08:31:16.158963+0000","last_clean":"2024-01-20T08:31:16.158963+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:31:16.158963+0000","last_undegraded":"2024-01-20T08:31:16.158963+0000","last_fullsized":"2024-01-20T08:31:16.158963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.089892+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:31:16.089892+0000","last_peered":"2024-01-20T08:31:16.089892+0000","last_clean":"2024-01-20T08:31:16.089892+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:31:16.089892+0000","last_undegraded":"2024-01-20T08:31:16.089892+0000","last_fullsized":"2024-01-20T08:31:16.089892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.090330+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:31:16.090330+0000","last_peered":"2024-01-20T08:31:16.090330+0000","last_clean":"2024-01-20T08:31:16.090330+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:31:16.090330+0000","last_undegraded":"2024-01-20T08:31:16.090330+0000","last_fullsized":"2024-01-20T08:31:16.090330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087503+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:31:16.087503+0000","last_peered":"2024-01-20T08:31:16.087503+0000","last_clean":"2024-01-20T08:31:16.087503+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:31:16.087503+0000","last_undegraded":"2024-01-20T08:31:16.087503+0000","last_fullsized":"2024-01-20T08:31:16.087503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087327+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:31:16.087327+0000","last_peered":"2024-01-20T08:31:16.087327+0000","last_clean":"2024-01-20T08:31:16.087327+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:31:16.087327+0000","last_undegraded":"2024-01-20T08:31:16.087327+0000","last_fullsized":"2024-01-20T08:31:16.087327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087526+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:31:16.087526+0000","last_peered":"2024-01-20T08:31:16.087526+0000","last_clean":"2024-01-20T08:31:16.087526+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:31:16.087526+0000","last_undegraded":"2024-01-20T08:31:16.087526+0000","last_fullsized":"2024-01-20T08:31:16.087526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.173025+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:31:16.173025+0000","last_peered":"2024-01-20T08:31:16.173025+0000","last_clean":"2024-01-20T08:31:16.173025+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:31:16.173025+0000","last_undegraded":"2024-01-20T08:31:16.173025+0000","last_fullsized":"2024-01-20T08:31:16.173025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159022+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:31:16.159022+0000","last_peered":"2024-01-20T08:31:16.159022+0000","last_clean":"2024-01-20T08:31:16.159022+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:31:16.159022+0000","last_undegraded":"2024-01-20T08:31:16.159022+0000","last_fullsized":"2024-01-20T08:31:16.159022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087198+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:31:16.087198+0000","last_peered":"2024-01-20T08:31:16.087198+0000","last_clean":"2024-01-20T08:31:16.087198+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:31:16.087198+0000","last_undegraded":"2024-01-20T08:31:16.087198+0000","last_fullsized":"2024-01-20T08:31:16.087198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.170339+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:31:16.170339+0000","last_peered":"2024-01-20T08:31:16.170339+0000","last_clean":"2024-01-20T08:31:16.170339+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:31:16.170339+0000","last_undegraded":"2024-01-20T08:31:16.170339+0000","last_fullsized":"2024-01-20T08:31:16.170339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087386+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:31:16.087386+0000","last_peered":"2024-01-20T08:31:16.087386+0000","last_clean":"2024-01-20T08:31:16.087386+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:31:16.087386+0000","last_undegraded":"2024-01-20T08:31:16.087386+0000","last_fullsized":"2024-01-20T08:31:16.087386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.169135+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:31:16.169135+0000","last_peered":"2024-01-20T08:31:16.169135+0000","last_clean":"2024-01-20T08:31:16.169135+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:31:16.169135+0000","last_undegraded":"2024-01-20T08:31:16.169135+0000","last_fullsized":"2024-01-20T08:31:16.169135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159545+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:31:16.159545+0000","last_peered":"2024-01-20T08:31:16.159545+0000","last_clean":"2024-01-20T08:31:16.159545+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:31:16.159545+0000","last_undegraded":"2024-01-20T08:31:16.159545+0000","last_fullsized":"2024-01-20T08:31:16.159545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.159745+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:31:16.159745+0000","last_peered":"2024-01-20T08:31:16.159745+0000","last_clean":"2024-01-20T08:31:16.159745+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:31:16.159745+0000","last_undegraded":"2024-01-20T08:31:16.159745+0000","last_fullsized":"2024-01-20T08:31:16.159745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-01-20T08:31:16.087758+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:31:16.087758+0000","last_peered":"2024-01-20T08:31:16.087758+0000","last_clean":"2024-01-20T08:31:16.087758+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:31:16.087758+0000","last_undegraded":"2024-01-20T08:31:16.087758+0000","last_fullsized":"2024-01-20T08:31:16.087758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5243,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":9882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59,"ondisk_log_size":59,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":103963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":694423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":694401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":104567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:31:41.583 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-20T08:31:41.584 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-20T08:31:41.584 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-20T08:31:41.584 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-20T08:31:41.584 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-20T08:31:41.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.731+0000 7f611f696640 1 Processor -- start 2024-01-20T08:31:41.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.731+0000 7f611f696640 1 -- start start 2024-01-20T08:31:41.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.732+0000 7f611f696640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 0x7f61180cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.732+0000 7f611f696640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f61180ca090 0x7f61180ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.732+0000 7f611f696640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f611815d6f0 0x7f61180cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.732+0000 7f611f696640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611815a560 con 0x7f611815d6f0 2024-01-20T08:31:41.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.732+0000 7f611f696640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6118153eb0 con 0x7f61180ca090 2024-01-20T08:31:41.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.732+0000 7f611f696640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611815dc80 con 0x7f6118153ad0 2024-01-20T08:31:41.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.733+0000 7f611d432640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 0x7f61180cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.733+0000 7f611cc31640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f61180ca090 0x7f61180ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.733+0000 7f611dc33640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f611815d6f0 0x7f61180cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.733+0000 7f611d432640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 0x7f61180cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36332/0 (socket says 172.21.15.87:36332) 2024-01-20T08:31:41.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.733+0000 7f611d432640 1 -- 172.21.15.87:0/922400416 learned_addr learned my addr 172.21.15.87:0/922400416 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:41.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.735+0000 7f611d432640 1 -- 172.21.15.87:0/922400416 >> v2:172.21.15.116:3300/0 conn(0x7f61180ca090 msgr2=0x7f61180ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:41.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.735+0000 7f611d432640 1 --2- 172.21.15.87:0/922400416 >> v2:172.21.15.116:3300/0 conn(0x7f61180ca090 0x7f61180ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.735+0000 7f611d432640 1 -- 172.21.15.87:0/922400416 >> v2:172.21.15.87:3300/0 conn(0x7f611815d6f0 msgr2=0x7f61180cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:41.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.735+0000 7f611d432640 1 --2- 172.21.15.87:0/922400416 >> v2:172.21.15.87:3300/0 conn(0x7f611815d6f0 0x7f61180cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.735+0000 7f611d432640 1 -- 172.21.15.87:0/922400416 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180cb760 con 0x7f6118153ad0 2024-01-20T08:31:41.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.737+0000 7f611d432640 1 --2- 172.21.15.87:0/922400416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 0x7f61180cb220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f611808c380 tx=0x7f611402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423eac5bb851ef7f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:41.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f610ffff640 1 -- 172.21.15.87:0/922400416 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114031870 con 0x7f6118153ad0 2024-01-20T08:31:41.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f610ffff640 1 -- 172.21.15.87:0/922400416 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61140319d0 con 0x7f6118153ad0 2024-01-20T08:31:41.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f610ffff640 1 -- 172.21.15.87:0/922400416 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114031d60 con 0x7f6118153ad0 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 -- 172.21.15.87:0/922400416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 msgr2=0x7f61180cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 --2- 172.21.15.87:0/922400416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 0x7f61180cb220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f611808c380 tx=0x7f611402b910 comp rx=0 tx=0).stop 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 -- 172.21.15.87:0/922400416 shutdown_connections 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 --2- 172.21.15.87:0/922400416 >> v2:172.21.15.87:3300/0 conn(0x7f611815d6f0 0x7f61180cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 --2- 172.21.15.87:0/922400416 >> v2:172.21.15.116:3300/0 conn(0x7f61180ca090 0x7f61180ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 --2- 172.21.15.87:0/922400416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6118153ad0 0x7f61180cb220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 -- 172.21.15.87:0/922400416 >> 172.21.15.87:0/922400416 conn(0x7f611808af90 msgr2=0x7f611808b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 -- 172.21.15.87:0/922400416 shutdown_connections 2024-01-20T08:31:41.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.741+0000 7f611f696640 1 -- 172.21.15.87:0/922400416 wait complete. 2024-01-20T08:31:41.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 Processor -- start 2024-01-20T08:31:41.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 -- start start 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61180ca090 0x7f6118154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6118153ad0 0x7f611814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 0x7f611815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181509a0 con 0x7f6118153ad0 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611814fc80 con 0x7f61180ca090 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.743+0000 7f611f696640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611815b690 con 0x7f611815d6f0 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.744+0000 7f611cc31640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6118153ad0 0x7f611814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.744+0000 7f611dc33640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 0x7f611815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.744+0000 7f611d432640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61180ca090 0x7f6118154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.744+0000 7f611dc33640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 0x7f611815bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36342/0 (socket says 172.21.15.87:36342) 2024-01-20T08:31:41.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.744+0000 7f611dc33640 1 -- 172.21.15.87:0/3162627340 learned_addr learned my addr 172.21.15.87:0/3162627340 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:41.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.747+0000 7f611dc33640 1 -- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61180ca090 msgr2=0x7f6118154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.747+0000 7f611dc33640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61180ca090 0x7f6118154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.747+0000 7f611dc33640 1 -- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6118153ad0 msgr2=0x7f611814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:41.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.747+0000 7f611dc33640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6118153ad0 0x7f611814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:41.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.747+0000 7f611dc33640 1 -- 172.21.15.87:0/3162627340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140094d0 con 0x7f611815d6f0 2024-01-20T08:31:41.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.750+0000 7f611dc33640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 0x7f611815bb20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f611000efc0 tx=0x7f611000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:41.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.753+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110009ba0 con 0x7f611815d6f0 2024-01-20T08:31:41.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.753+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611000f040 con 0x7f611815d6f0 2024-01-20T08:31:41.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.753+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611815cb40 con 0x7f611815d6f0 2024-01-20T08:31:41.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.753+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100146c0 con 0x7f611815d6f0 2024-01-20T08:31:41.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.754+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611815ad60 con 0x7f611815d6f0 2024-01-20T08:31:41.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.755+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6118074290 con 0x7f611815d6f0 2024-01-20T08:31:41.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.773+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6110014820 con 0x7f611815d6f0 2024-01-20T08:31:41.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.773+0000 7f610dffb640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f60fc06af20 0x7f60fc06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:41.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.773+0000 7f611d432640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f60fc06af20 0x7f60fc06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:41.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.773+0000 7f611d432640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f60fc06af20 0x7f60fc06d3e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6114036000 tx=0x7f611403c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:41.775+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61100593e0 con 0x7f611815d6f0 2024-01-20T08:31:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.009+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6110055720 con 0x7f611815d6f0 2024-01-20T08:31:42.038 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-01-20T08:31:42.035+0000 f5d6640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-01-20T08:31:42.038 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-01-20T08:31:42.035+0000 f5d6640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-01-20T08:31:42.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.235+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f61180ca8f0 con 0x7f611815d6f0 2024-01-20T08:31:42.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.376+0000 7f610dffb640 1 -- 172.21.15.87:0/3162627340 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v7) v1 ==== 74+0+170768 (secure 0 0 0) 0x7f6110018070 con 0x7f611815d6f0 2024-01-20T08:31:42.382 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f60fc06af20 msgr2=0x7f60fc06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f60fc06af20 0x7f60fc06d3e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6114036000 tx=0x7f611403c040 comp rx=0 tx=0).stop 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 msgr2=0x7f611815bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 0x7f611815bb20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f611000efc0 tx=0x7f611000c490 comp rx=0 tx=0).stop 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 shutdown_connections 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f60fc06af20 0x7f60fc06d3e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f611815d6f0 0x7f611815bb20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6118153ad0 0x7f611814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 --2- 172.21.15.87:0/3162627340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61180ca090 0x7f6118154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 >> 172.21.15.87:0/3162627340 conn(0x7f611808af90 msgr2=0x7f61181512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 shutdown_connections 2024-01-20T08:31:42.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.379+0000 7f611f696640 1 -- 172.21.15.87:0/3162627340 wait complete. 2024-01-20T08:31:42.393 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":7,"active_gid":4154,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2055452307},{"type":"v1","addr":"172.21.15.87:6833","nonce":2055452307}]},"active_addr":"172.21.15.87:6833/2055452307","active_change":"2024-01-20T08:30:41.833276+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":12,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3781369618}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":2987780804}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":510700154}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":240607993}]}]} 2024-01-20T08:31:42.397 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-20T08:31:42.397 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-20T08:31:42.397 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:42.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.498+0000 7f227e35a640 1 Processor -- start 2024-01-20T08:31:42.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.498+0000 7f227e35a640 1 -- start start 2024-01-20T08:31:42.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227e35a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2278153ad0 0x7f22780cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227e35a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 0x7f22780ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227e35a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f227815d6f0 0x7f22780cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227e35a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227815a560 con 0x7f2278153ad0 2024-01-20T08:31:42.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227e35a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2278153eb0 con 0x7f227815d6f0 2024-01-20T08:31:42.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227e35a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227815dc80 con 0x7f22780ca090 2024-01-20T08:31:42.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f22777fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 0x7f22780ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:42.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.499+0000 7f227c8f7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f227815d6f0 0x7f22780cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:42.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.500+0000 7f22777fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 0x7f22780ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36356/0 (socket says 172.21.15.87:36356) 2024-01-20T08:31:42.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.500+0000 7f227c8f7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f227815d6f0 0x7f22780cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50084/0 (socket says 172.21.15.87:50084) 2024-01-20T08:31:42.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.500+0000 7f22777fe640 1 -- 172.21.15.87:0/761261237 learned_addr learned my addr 172.21.15.87:0/761261237 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:42.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.502+0000 7f227c8f7640 1 -- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 msgr2=0x7f22780ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:42.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.502+0000 7f227c8f7640 1 --2- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 0x7f22780ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.502+0000 7f227c8f7640 1 -- 172.21.15.87:0/761261237 >> v2:172.21.15.87:3300/0 conn(0x7f2278153ad0 msgr2=0x7f22780cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:42.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.502+0000 7f227c8f7640 1 --2- 172.21.15.87:0/761261237 >> v2:172.21.15.87:3300/0 conn(0x7f2278153ad0 0x7f22780cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.502+0000 7f227c8f7640 1 -- 172.21.15.87:0/761261237 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780cb760 con 0x7f227815d6f0 2024-01-20T08:31:42.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.502+0000 7f22777fe640 1 --2- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 0x7f22780ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:42.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.504+0000 7f227c8f7640 1 --2- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22780cf400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f227808bab0 tx=0x7f227002be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2966bca80a3d2e54 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f2276ffd640 1 -- 172.21.15.87:0/761261237 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227002cbd0 con 0x7f227815d6f0 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f2276ffd640 1 -- 172.21.15.87:0/761261237 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227002cd30 con 0x7f227815d6f0 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f2276ffd640 1 -- 172.21.15.87:0/761261237 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227002cea0 con 0x7f227815d6f0 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 -- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 msgr2=0x7f22780cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 --2- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22780cf400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f227808bab0 tx=0x7f227002be20 comp rx=0 tx=0).stop 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 -- 172.21.15.87:0/761261237 shutdown_connections 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 --2- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22780cf400 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 --2- 172.21.15.87:0/761261237 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22780ca090 0x7f22780ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 --2- 172.21.15.87:0/761261237 >> v2:172.21.15.87:3300/0 conn(0x7f2278153ad0 0x7f22780cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 -- 172.21.15.87:0/761261237 >> 172.21.15.87:0/761261237 conn(0x7f227808af90 msgr2=0x7f227808b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:42.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 -- 172.21.15.87:0/761261237 shutdown_connections 2024-01-20T08:31:42.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.508+0000 7f227e35a640 1 -- 172.21.15.87:0/761261237 wait complete. 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.510+0000 7f227e35a640 1 Processor -- start 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.510+0000 7f227e35a640 1 -- start start 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227e35a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22780ca090 0x7f227814b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227e35a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2278153ad0 0x7f2278158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227e35a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22781496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227e35a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227815bd80 con 0x7f22780ca090 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227e35a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227815b790 con 0x7f227815d6f0 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227e35a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2278149be0 con 0x7f2278153ad0 2024-01-20T08:31:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227c8f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22781496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f22777fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2278153ad0 0x7f2278158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227c8f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22781496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50096/0 (socket says 172.21.15.87:50096) 2024-01-20T08:31:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.511+0000 7f227c8f7640 1 -- 172.21.15.87:0/2128496860 learned_addr learned my addr 172.21.15.87:0/2128496860 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:42.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.515+0000 7f227c8f7640 1 -- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2278153ad0 msgr2=0x7f2278158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:42.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.515+0000 7f227c8f7640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2278153ad0 0x7f2278158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.515+0000 7f227c8f7640 1 -- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22780ca090 msgr2=0x7f227814b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:42.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.515+0000 7f227c8f7640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22780ca090 0x7f227814b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:42.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.515+0000 7f227c8f7640 1 -- 172.21.15.87:0/2128496860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0094d0 con 0x7f227815d6f0 2024-01-20T08:31:42.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.517+0000 7f227c8f7640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22781496a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2270009e80 tx=0x7f2270004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.521+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227002cea0 con 0x7f227815d6f0 2024-01-20T08:31:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.521+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270040650 con 0x7f227815d6f0 2024-01-20T08:31:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.521+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270040a00 con 0x7f227815d6f0 2024-01-20T08:31:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.521+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2270009a70 con 0x7f227815d6f0 2024-01-20T08:31:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.521+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2278158460 con 0x7f227815d6f0 2024-01-20T08:31:42.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.522+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2278074290 con 0x7f227815d6f0 2024-01-20T08:31:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.821+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2270040b60 con 0x7f227815d6f0 2024-01-20T08:31:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.821+0000 7f2274ff9640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f226406af50 0x7f226406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.821+0000 7f2277fff640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f226406af50 0x7f226406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.821+0000 7f2277fff640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f226406af50 0x7f226406d410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2268009710 tx=0x7f2268006c90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.824+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f227007ba10 con 0x7f227815d6f0 2024-01-20T08:31:42.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:42.834+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2270077650 con 0x7f227815d6f0 2024-01-20T08:31:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.018+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22780ca8f0 con 0x7f227815d6f0 2024-01-20T08:31:43.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.080+0000 7f2274ff9640 1 -- 172.21.15.87:0/2128496860 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18446 (secure 0 0 0) 0x7f2270009450 con 0x7f227815d6f0 2024-01-20T08:31:43.083 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:43.084 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:42.581565+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f226406af50 msgr2=0x7f226406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f226406af50 0x7f226406d410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2268009710 tx=0x7f2268006c90 comp rx=0 tx=0).stop 2024-01-20T08:31:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 msgr2=0x7f22781496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22781496a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2270009e80 tx=0x7f2270004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 shutdown_connections 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f226406af50 0x7f226406d410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227815d6f0 0x7f22781496a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2278153ad0 0x7f2278158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 --2- 172.21.15.87:0/2128496860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22780ca090 0x7f227814b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 >> 172.21.15.87:0/2128496860 conn(0x7f227808af90 msgr2=0x7f22780cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 shutdown_connections 2024-01-20T08:31:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.082+0000 7f227e35a640 1 -- 172.21.15.87:0/2128496860 wait complete. 2024-01-20T08:31:43.097 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-20T08:31:43.097 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:43.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.194+0000 7fb729489640 1 Processor -- start 2024-01-20T08:31:43.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.194+0000 7fb729489640 1 -- start start 2024-01-20T08:31:43.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb729489640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb724150780 0x7fb72415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb729489640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb72406bf30 0x7fb72406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb729489640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7240ca960 con 0x7fb72406b750 2024-01-20T08:31:43.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb729489640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7240c6c50 con 0x7fb724150780 2024-01-20T08:31:43.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb729489640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72415a820 con 0x7fb72406bf30 2024-01-20T08:31:43.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb7237fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb72406b750 0x7fb7240c7890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38946/0 (socket says 172.21.15.87:38946) 2024-01-20T08:31:43.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb7237fe640 1 -- 172.21.15.87:0/26958129 learned_addr learned my addr 172.21.15.87:0/26958129 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:43.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb722ffd640 1 --2- 172.21.15.87:0/26958129 >> v2:172.21.15.87:3300/0 conn(0x7fb724150780 0x7fb72415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.195+0000 7fb7227fc640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb72406bf30 0x7fb72406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb721ffb640 1 -- 172.21.15.87:0/26958129 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519155011 0 0) 0x7fb7240ca960 con 0x7fb72406b750 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb721ffb640 1 -- 172.21.15.87:0/26958129 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb70c003520 con 0x7fb72406b750 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb722ffd640 1 -- 172.21.15.87:0/26958129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb72406bf30 msgr2=0x7fb72406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb722ffd640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb72406bf30 0x7fb72406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb722ffd640 1 -- 172.21.15.87:0/26958129 >> v1:172.21.15.116:6789/0 conn(0x7fb72406b750 legacy=0x7fb7240c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb722ffd640 1 -- 172.21.15.87:0/26958129 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240cadc0 con 0x7fb724150780 2024-01-20T08:31:43.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.198+0000 7fb7227fc640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb72406bf30 0x7fb72406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:43.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.200+0000 7fb722ffd640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb724150780 0x7fb72415a2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb714009d90 tx=0x7fb71402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c625fc8ab7500e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.203+0000 7fb721ffb640 1 -- 172.21.15.87:0/26958129 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71402cce0 con 0x7fb724150780 2024-01-20T08:31:43.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.203+0000 7fb721ffb640 1 -- 172.21.15.87:0/26958129 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71402ce40 con 0x7fb724150780 2024-01-20T08:31:43.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.203+0000 7fb721ffb640 1 -- 172.21.15.87:0/26958129 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714034660 con 0x7fb724150780 2024-01-20T08:31:43.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 -- 172.21.15.87:0/26958129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb724150780 msgr2=0x7fb72415a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb724150780 0x7fb72415a2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb714009d90 tx=0x7fb71402bf30 comp rx=0 tx=0).stop 2024-01-20T08:31:43.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 -- 172.21.15.87:0/26958129 shutdown_connections 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb72406bf30 0x7fb72406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 --2- 172.21.15.87:0/26958129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb724150780 0x7fb72415a2e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 -- 172.21.15.87:0/26958129 >> 172.21.15.87:0/26958129 conn(0x7fb7240cb900 msgr2=0x7fb7240cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 -- 172.21.15.87:0/26958129 shutdown_connections 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.204+0000 7fb729489640 1 -- 172.21.15.87:0/26958129 wait complete. 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 Processor -- start 2024-01-20T08:31:43.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 -- start start 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb72406b750 0x7fb7241598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb72406bf30 0x7fb7240b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb724150780 0x7fb724150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724152670 con 0x7fb72406bf30 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724152950 con 0x7fb72406b750 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.206+0000 7fb729489640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb724153a90 con 0x7fb724150780 2024-01-20T08:31:43.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.207+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb72406bf30 0x7fb7240b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.207+0000 7fb722ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb72406b750 0x7fb7241598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.207+0000 7fb7237fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb724150780 0x7fb724150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.207+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb72406bf30 0x7fb7240b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37698/0 (socket says 172.21.15.87:37698) 2024-01-20T08:31:43.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.207+0000 7fb7227fc640 1 -- 172.21.15.87:0/1820199443 learned_addr learned my addr 172.21.15.87:0/1820199443 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:43.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.210+0000 7fb722ffd640 1 -- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb724150780 msgr2=0x7fb724150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.210+0000 7fb722ffd640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb724150780 0x7fb724150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.210+0000 7fb722ffd640 1 -- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb72406bf30 msgr2=0x7fb7240b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.210+0000 7fb722ffd640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb72406bf30 0x7fb7240b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.211+0000 7fb722ffd640 1 -- 172.21.15.87:0/1820199443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718009400 con 0x7fb72406b750 2024-01-20T08:31:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.213+0000 7fb722ffd640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb72406b750 0x7fb7241598e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb714033000 tx=0x7fb714004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.216+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71403f680 con 0x7fb72406b750 2024-01-20T08:31:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.216+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71403f800 con 0x7fb72406b750 2024-01-20T08:31:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.216+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb714009a70 con 0x7fb72406b750 2024-01-20T08:31:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.216+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71403fbb0 con 0x7fb72406b750 2024-01-20T08:31:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.216+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7241537b0 con 0x7fb72406b750 2024-01-20T08:31:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.217+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb724074290 con 0x7fb72406b750 2024-01-20T08:31:43.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.459+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb71403b040 con 0x7fb72406b750 2024-01-20T08:31:43.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.460+0000 7fb70b7fe640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb71006af20 0x7fb71006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.460+0000 7fb7227fc640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb71006af20 0x7fb71006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.460+0000 7fb7227fc640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb71006af20 0x7fb71006d3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb72414bdc0 tx=0x7fb718009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.463+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fb71407b150 con 0x7fb72406b750 2024-01-20T08:31:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.473+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb714077490 con 0x7fb72406b750 2024-01-20T08:31:43.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.656+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb724153e40 con 0x7fb72406b750 2024-01-20T08:31:43.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.753+0000 7fb70b7fe640 1 -- 172.21.15.87:0/1820199443 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18446 (secure 0 0 0) 0x7fb714039090 con 0x7fb72406b750 2024-01-20T08:31:43.757 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:43.757 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:42.581565+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb71006af20 msgr2=0x7fb71006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb71006af20 0x7fb71006d3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb72414bdc0 tx=0x7fb718009190 comp rx=0 tx=0).stop 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb72406b750 msgr2=0x7fb7241598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb72406b750 0x7fb7241598e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb714033000 tx=0x7fb714004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 shutdown_connections 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fb71006af20 0x7fb71006d3e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb724150780 0x7fb724150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb72406bf30 0x7fb7240b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 --2- 172.21.15.87:0/1820199443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb72406b750 0x7fb7241598e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 >> 172.21.15.87:0/1820199443 conn(0x7fb7240cb900 msgr2=0x7fb7240c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:43.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 shutdown_connections 2024-01-20T08:31:43.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.756+0000 7fb729489640 1 -- 172.21.15.87:0/1820199443 wait complete. 2024-01-20T08:31:43.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-20T08:31:43.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-20T08:31:43.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-20T08:31:43.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-20T08:31:43.772 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-20T08:31:43.772 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-20T08:31:43.772 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-20T08:31:43.773 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.960+0000 7f5a0359e640 1 Processor -- start 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.960+0000 7f5a0359e640 1 -- start start 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.961+0000 7f5a0359e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a040c5ad0 0x7f5a040c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.961+0000 7f5a0359e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a040c93e0 con 0x7f5a04157b70 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.961+0000 7f5a0359e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a040ca260 con 0x7f5a0415e160 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.962+0000 7f5a0359e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041611c0 con 0x7f5a040c5ad0 2024-01-20T08:31:43.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.962+0000 7f5a02d9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5a0415e160 0x7f5a040c7650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38950/0 (socket says 172.21.15.87:38950) 2024-01-20T08:31:43.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.962+0000 7f5a02d9d640 1 -- 172.21.15.87:0/705528225 learned_addr learned my addr 172.21.15.87:0/705528225 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:43.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.962+0000 7f327cc9b640 1 Processor -- start 2024-01-20T08:31:43.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.963+0000 7fd09bfd7640 1 Processor -- start 2024-01-20T08:31:43.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.962+0000 7f327cc9b640 1 -- start start 2024-01-20T08:31:43.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.964+0000 7f327cc9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f327004b4f0 0x7f32700dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.964+0000 7f327cc9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 0x7f3270005e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.964+0000 7f327cc9b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3270007700 con 0x7f32700063c0 2024-01-20T08:31:43.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.964+0000 7f327cc9b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32700068a0 con 0x7f327004b4f0 2024-01-20T08:31:43.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.964+0000 7f327cc9b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3270006b80 con 0x7f327004b8d0 2024-01-20T08:31:43.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.964+0000 7f5a01d9b640 1 --2- 172.21.15.87:0/705528225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a040c5ad0 0x7f5a040c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.963+0000 7fd09bfd7640 1 -- start start 2024-01-20T08:31:43.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7f5a0159a640 1 -- 172.21.15.87:0/705528225 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322117422 0 0) 0x7f5a040ca260 con 0x7f5a0415e160 2024-01-20T08:31:43.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7f5a0159a640 1 -- 172.21.15.87:0/705528225 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59ec003520 con 0x7f5a0415e160 2024-01-20T08:31:43.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7fd09bfd7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0940c7e10 0x7fd094155570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7fd09bfd7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd094155b60 con 0x7fd094070d00 2024-01-20T08:31:43.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7fd09bfd7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd094155e20 con 0x7fd09414f7b0 2024-01-20T08:31:43.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7fd09bfd7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd094156100 con 0x7fd0940c7e10 2024-01-20T08:31:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.965+0000 7f7382bc2640 1 Processor -- start 2024-01-20T08:31:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.966+0000 7f7382bc2640 1 -- start start 2024-01-20T08:31:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.966+0000 7fd099572640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd094070d00 0x7fd0940c7700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34154/0 (socket says 172.21.15.87:34154) 2024-01-20T08:31:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.966+0000 7fd099572640 1 -- 172.21.15.87:0/295144659 learned_addr learned my addr 172.21.15.87:0/295144659 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f7382bc2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f737c157b70 0x7f737c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f7382bc2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f737c0719d0 0x7f737c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f7382bc2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737c157f50 con 0x7f737c071200 2024-01-20T08:31:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f7382bc2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c161080 con 0x7f737c0719d0 2024-01-20T08:31:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f7382bc2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c15e7b0 con 0x7f737c157b70 2024-01-20T08:31:43.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f3275d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 0x7f3270005e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f3275d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 0x7f3270005e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36376/0 (socket says 172.21.15.87:36376) 2024-01-20T08:31:43.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7f3275d9b640 1 -- 172.21.15.87:0/860297708 learned_addr learned my addr 172.21.15.87:0/860297708 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:43.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.967+0000 7fd09a574640 1 --2- 172.21.15.87:0/295144659 >> v2:172.21.15.116:3300/0 conn(0x7fd0940c7e10 0x7fd094155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.969+0000 7f5a01d9b640 1 -- 172.21.15.87:0/705528225 >> v1:172.21.15.116:6789/0 conn(0x7f5a0415e160 legacy=0x7f5a040c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.969+0000 7f5a01d9b640 1 -- 172.21.15.87:0/705528225 >> v1:172.21.15.87:6789/0 conn(0x7f5a04157b70 legacy=0x7f5a041615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.969+0000 7f867544f640 1 Processor -- start 2024-01-20T08:31:43.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.969+0000 7f5a01d9b640 1 -- 172.21.15.87:0/705528225 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0415e5f0 con 0x7f5a040c5ad0 2024-01-20T08:31:43.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.969+0000 7f867544f640 1 -- start start 2024-01-20T08:31:43.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.971+0000 7fd09a574640 1 -- 172.21.15.87:0/295144659 >> v1:172.21.15.116:6789/0 conn(0x7fd09414f7b0 legacy=0x7fd0940caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:43.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.971+0000 7fd09a574640 1 -- 172.21.15.87:0/295144659 >> v1:172.21.15.87:6789/0 conn(0x7fd094070d00 legacy=0x7fd0940c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.971+0000 7fd09a574640 1 -- 172.21.15.87:0/295144659 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0941563e0 con 0x7fd0940c7e10 2024-01-20T08:31:43.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.972+0000 7f867544f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 0x7f867015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.972+0000 7f867544f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86701612c0 0x7f86701685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.972+0000 7f867544f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867015e130 con 0x7f86700c8c00 2024-01-20T08:31:43.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.972+0000 7f867544f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86700cbbd0 con 0x7f86701612c0 2024-01-20T08:31:43.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.972+0000 7f867544f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670161800 con 0x7f86700c8820 2024-01-20T08:31:43.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.973+0000 7f3275d9b640 1 -- 172.21.15.87:0/860297708 >> v2:172.21.15.87:3300/0 conn(0x7f327004b4f0 msgr2=0x7f32700dac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:43.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.973+0000 7f3275d9b640 1 --2- 172.21.15.87:0/860297708 >> v2:172.21.15.87:3300/0 conn(0x7f327004b4f0 0x7f32700dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.973+0000 7f3275d9b640 1 -- 172.21.15.87:0/860297708 >> v1:172.21.15.116:6789/0 conn(0x7f32700063c0 legacy=0x7f3270034a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.973+0000 7f3275d9b640 1 -- 172.21.15.87:0/860297708 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270006e60 con 0x7f327004b8d0 2024-01-20T08:31:43.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.975+0000 7f5a01d9b640 1 --2- 172.21.15.87:0/705528225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a040c5ad0 0x7f5a040c8dc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59f80097d0 tx=0x7f59f802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4c44f375575739 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.975+0000 7f73813bf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f737c0719d0 0x7f737c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.976+0000 7f3275d9b640 1 --2- 172.21.15.87:0/860297708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 0x7f3270005e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3278073590 tx=0x7f32780a6dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b0bab402a1cfde server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.977+0000 7f866f7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86701612c0 0x7f86701685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.978+0000 7fd09a574640 1 --2- 172.21.15.87:0/295144659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0940c7e10 0x7fd094155570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd0840097a0 tx=0x7fd08402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755f4563ae0dfb8c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.978+0000 7f8667fff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f86700c8c00 0x7f86700c5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34162/0 (socket says 172.21.15.87:34162) 2024-01-20T08:31:43.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.978+0000 7f8667fff640 1 -- 172.21.15.87:0/3248405383 learned_addr learned my addr 172.21.15.87:0/3248405383 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:43.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.978+0000 7f7381bc0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f737c157b70 0x7f737c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.978+0000 7f73813bf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f737c0719d0 0x7f737c1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37720/0 (socket says 172.21.15.87:37720) 2024-01-20T08:31:43.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.978+0000 7f73813bf640 1 -- 172.21.15.87:0/4279301967 learned_addr learned my addr 172.21.15.87:0/4279301967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:43.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.982+0000 7f866f7fe640 1 -- 172.21.15.87:0/3248405383 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 msgr2=0x7f867015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:43.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.982+0000 7f866f7fe640 1 --2- 172.21.15.87:0/3248405383 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 0x7f867015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.982+0000 7f866f7fe640 1 -- 172.21.15.87:0/3248405383 >> v1:172.21.15.87:6789/0 conn(0x7f86700c8c00 legacy=0x7f86700c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.982+0000 7f866f7fe640 1 -- 172.21.15.87:0/3248405383 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670161ae0 con 0x7f86701612c0 2024-01-20T08:31:43.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.984+0000 7f7381bc0640 1 -- 172.21.15.87:0/4279301967 >> v2:172.21.15.87:3300/0 conn(0x7f737c0719d0 msgr2=0x7f737c1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:43.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.984+0000 7f7381bc0640 1 --2- 172.21.15.87:0/4279301967 >> v2:172.21.15.87:3300/0 conn(0x7f737c0719d0 0x7f737c1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.984+0000 7f7381bc0640 1 -- 172.21.15.87:0/4279301967 >> v1:172.21.15.116:6789/0 conn(0x7f737c071200 legacy=0x7f737c0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:43.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.984+0000 7f7381bc0640 1 -- 172.21.15.87:0/4279301967 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c072090 con 0x7f737c157b70 2024-01-20T08:31:43.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.985+0000 7f5a0159a640 1 -- 172.21.15.87:0/705528225 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8031890 con 0x7f5a040c5ad0 2024-01-20T08:31:43.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.985+0000 7f5a0159a640 1 -- 172.21.15.87:0/705528225 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f80319f0 con 0x7f5a040c5ad0 2024-01-20T08:31:43.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.985+0000 7f5a0159a640 1 -- 172.21.15.87:0/705528225 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8031d80 con 0x7f5a040c5ad0 2024-01-20T08:31:43.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.986+0000 7f327559a640 1 -- 172.21.15.87:0/860297708 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780a7ca0 con 0x7f327004b8d0 2024-01-20T08:31:43.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.986+0000 7f327559a640 1 -- 172.21.15.87:0/860297708 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32780a7e40 con 0x7f327004b8d0 2024-01-20T08:31:43.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.986+0000 7f327559a640 1 -- 172.21.15.87:0/860297708 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780af640 con 0x7f327004b8d0 2024-01-20T08:31:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.986+0000 7f5a0359e640 1 -- 172.21.15.87:0/705528225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a040c5ad0 msgr2=0x7f5a040c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.986+0000 7f5a0359e640 1 --2- 172.21.15.87:0/705528225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a040c5ad0 0x7f5a040c8dc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59f80097d0 tx=0x7f59f802b930 comp rx=0 tx=0).stop 2024-01-20T08:31:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f5a0359e640 1 -- 172.21.15.87:0/705528225 shutdown_connections 2024-01-20T08:31:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f5a0359e640 1 --2- 172.21.15.87:0/705528225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a040c5ad0 0x7f5a040c8dc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f5a0359e640 1 -- 172.21.15.87:0/705528225 >> 172.21.15.87:0/705528225 conn(0x7f5a040ca970 msgr2=0x7f5a040cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f5a0359e640 1 -- 172.21.15.87:0/705528225 shutdown_connections 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 -- 172.21.15.87:0/860297708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 msgr2=0x7f3270005e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 --2- 172.21.15.87:0/860297708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 0x7f3270005e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3278073590 tx=0x7f32780a6dd0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 -- 172.21.15.87:0/860297708 shutdown_connections 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 --2- 172.21.15.87:0/860297708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b8d0 0x7f3270005e80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 --2- 172.21.15.87:0/860297708 >> v2:172.21.15.87:3300/0 conn(0x7f327004b4f0 0x7f32700dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 -- 172.21.15.87:0/860297708 >> 172.21.15.87:0/860297708 conn(0x7f3270050170 msgr2=0x7f3270050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 -- 172.21.15.87:0/860297708 shutdown_connections 2024-01-20T08:31:43.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f327cc9b640 1 -- 172.21.15.87:0/860297708 wait complete. 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.988+0000 7fd098d71640 1 -- 172.21.15.87:0/295144659 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084030610 con 0x7fd0940c7e10 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.988+0000 7fd098d71640 1 -- 172.21.15.87:0/295144659 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084030770 con 0x7fd0940c7e10 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 -- 172.21.15.87:0/295144659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0940c7e10 msgr2=0x7fd094155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 --2- 172.21.15.87:0/295144659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0940c7e10 0x7fd094155570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd0840097a0 tx=0x7fd08402b960 comp rx=0 tx=0).stop 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 -- 172.21.15.87:0/295144659 shutdown_connections 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 --2- 172.21.15.87:0/295144659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0940c7e10 0x7fd094155570 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:43.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 -- 172.21.15.87:0/295144659 >> 172.21.15.87:0/295144659 conn(0x7fd09406e8c0 msgr2=0x7fd09406eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.987+0000 7f5a0359e640 1 -- 172.21.15.87:0/705528225 wait complete. 2024-01-20T08:31:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7f5a0359e640 1 Processor -- start 2024-01-20T08:31:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 -- 172.21.15.87:0/295144659 shutdown_connections 2024-01-20T08:31:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7f5a0359e640 1 -- start start 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f5a0359e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a040c5ad0 0x7f5a040c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f5a0359e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a04157b70 0x7f5a04156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f5a0359e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a0415e160 0x7f5a041565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f5a0359e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0412c0a0 con 0x7f5a040c5ad0 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f5a0359e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04156ae0 con 0x7f5a0415e160 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f5a0359e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0412bb00 con 0x7f5a04157b70 2024-01-20T08:31:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f7381bc0640 1 --2- 172.21.15.87:0/4279301967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 0x7f737c15e270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f737c0706a0 tx=0x7f736c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d45380ae1b13a73 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:43.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7f327cc9b640 1 Processor -- start 2024-01-20T08:31:43.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7fd09bfd7640 1 -- 172.21.15.87:0/295144659 wait complete. 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.989+0000 7f327cc9b640 1 -- start start 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f327cc9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32700063c0 0x7f32700cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f327cc9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b4f0 0x7f32700d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f327cc9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f327004b8d0 0x7f32700d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.990+0000 7f327cc9b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700cf620 con 0x7f327004b8d0 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f327cc9b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700d2350 con 0x7f32700063c0 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f327cc9b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32700d04c0 con 0x7f327004b4f0 2024-01-20T08:31:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f5a02d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a0415e160 0x7f5a041565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f3275d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b4f0 0x7f32700d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7fd09bfd7640 1 Processor -- start 2024-01-20T08:31:43.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f5a01d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a04157b70 0x7f5a04156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7f327659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32700063c0 0x7f32700cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:43.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.992+0000 7f5a02d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a0415e160 0x7f5a041565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50146/0 (socket says 172.21.15.87:50146) 2024-01-20T08:31:43.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.992+0000 7f5a02d9d640 1 -- 172.21.15.87:0/2031402954 learned_addr learned my addr 172.21.15.87:0/2031402954 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:43.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.992+0000 7f3275d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b4f0 0x7f32700d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36388/0 (socket says 172.21.15.87:36388) 2024-01-20T08:31:43.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.992+0000 7f3275d9b640 1 -- 172.21.15.87:0/2702638459 learned_addr learned my addr 172.21.15.87:0/2702638459 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.991+0000 7fd09bfd7640 1 -- start start 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7fd09bfd7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd094070d00 0x7fd094148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7fd09bfd7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0940c7e10 0x7fd09414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7fd09bfd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd09414f7b0 0x7fd09414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7fd09bfd7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd094149840 con 0x7fd09414f7b0 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7fd09bfd7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd094149b20 con 0x7fd094070d00 2024-01-20T08:31:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7fd09bfd7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094149e00 con 0x7fd0940c7e10 2024-01-20T08:31:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.993+0000 7f866f7fe640 1 --2- 172.21.15.87:0/3248405383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86701612c0 0x7f86701685f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8668009d90 tx=0x7f866802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9945afcdf71da44f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.997+0000 7f3275d9b640 1 -- 172.21.15.87:0/2702638459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32700063c0 msgr2=0x7f32700cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.997+0000 7f3275d9b640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32700063c0 0x7f32700cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.997+0000 7f3275d9b640 1 -- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f327004b8d0 msgr2=0x7f32700d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.997+0000 7f3275d9b640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f327004b8d0 0x7f32700d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:43.997+0000 7f3275d9b640 1 -- 172.21.15.87:0/2702638459 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32780737e0 con 0x7f327004b4f0 2024-01-20T08:31:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.000+0000 7f5a01d9b640 1 -- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a0415e160 msgr2=0x7f5a041565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.000+0000 7f5a01d9b640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a0415e160 0x7f5a041565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.000+0000 7f5a01d9b640 1 -- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a040c5ad0 msgr2=0x7f5a040c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.000+0000 7f5a01d9b640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a040c5ad0 0x7f5a040c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.000+0000 7f5a01d9b640 1 -- 172.21.15.87:0/2031402954 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009400 con 0x7f5a04157b70 2024-01-20T08:31:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.002+0000 7f9ee7cee640 1 Processor -- start 2024-01-20T08:31:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.002+0000 7f9ee7cee640 1 -- start start 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.002+0000 7ff8e7741640 1 Processor -- start 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.003+0000 7ff8e7741640 1 -- start start 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.003+0000 7f9ee7cee640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ee006b1c0 0x7f9ee00c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.003+0000 7f9ee7cee640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ee015b740 0x7f9ee00c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.003+0000 7f9ee7cee640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee015aa10 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.003+0000 7f9ee7cee640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee006a8f0 con 0x7f9ee015b740 2024-01-20T08:31:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.003+0000 7f9ee7cee640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee006abb0 con 0x7f9ee006b1c0 2024-01-20T08:31:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7f230a246640 1 Processor -- start 2024-01-20T08:31:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7f230a246640 1 -- start start 2024-01-20T08:31:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7ff8e7741640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7ff8e7741640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8e015b740 0x7ff8e00c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7ff8e7741640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e015aa10 con 0x7ff8e00c6ca0 2024-01-20T08:31:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7ff8e7741640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e015b2b0 con 0x7ff8e015b740 2024-01-20T08:31:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.004+0000 7ff8e7741640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e006a8b0 con 0x7ff8e006b1c0 2024-01-20T08:31:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f230a246640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 0x7f23040703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f230a246640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2304156a30 0x7f2304156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f7380bbe640 1 -- 172.21.15.87:0/4279301967 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c02e070 con 0x7f737c157b70 2024-01-20T08:31:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f7380bbe640 1 -- 172.21.15.87:0/4279301967 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c0315a0 con 0x7f737c157b70 2024-01-20T08:31:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f7380bbe640 1 -- 172.21.15.87:0/4279301967 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c02e070 con 0x7f737c157b70 2024-01-20T08:31:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f230a246640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2304150b00 con 0x7f230406fff0 2024-01-20T08:31:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f230a246640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23040cbab0 con 0x7f2304156a30 2024-01-20T08:31:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f230a246640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2304070910 con 0x7f23040c6810 2024-01-20T08:31:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f5a01d9b640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a04157b70 0x7f5a04156060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59f8033000 tx=0x7f59f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.005+0000 7f3275d9b640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b4f0 0x7f32700d0fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3278067d30 tx=0x7f32780623e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 -- 172.21.15.87:0/4279301967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 msgr2=0x7f737c15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 --2- 172.21.15.87:0/4279301967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 0x7f737c15e270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f737c0706a0 tx=0x7f736c02b840 comp rx=0 tx=0).stop 2024-01-20T08:31:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 -- 172.21.15.87:0/4279301967 shutdown_connections 2024-01-20T08:31:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 --2- 172.21.15.87:0/4279301967 >> v2:172.21.15.87:3300/0 conn(0x7f737c0719d0 0x7f737c1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 --2- 172.21.15.87:0/4279301967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 0x7f737c15e270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 -- 172.21.15.87:0/4279301967 >> 172.21.15.87:0/4279301967 conn(0x7f737c0ca970 msgr2=0x7f737c0cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 -- 172.21.15.87:0/4279301967 shutdown_connections 2024-01-20T08:31:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7f7382bc2640 1 -- 172.21.15.87:0/4279301967 wait complete. 2024-01-20T08:31:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.006+0000 7fd099d73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd094070d00 0x7fd094148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.007+0000 7f7382bc2640 1 Processor -- start 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.007+0000 7f7382bc2640 1 -- start start 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.008+0000 7f7382bc2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f737c071200 0x7f737c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.008+0000 7f7382bc2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f737c0719d0 0x7f737c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.008+0000 7f7382bc2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 0x7f737c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.008+0000 7f7382bc2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c15d630 con 0x7f737c071200 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.008+0000 7f7382bc2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1e7eb0 con 0x7f737c157b70 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.008+0000 7f7382bc2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1f68a0 con 0x7f737c0719d0 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.010+0000 7fd099d73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd094070d00 0x7fd094148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50160/0 (socket says 172.21.15.87:50160) 2024-01-20T08:31:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.010+0000 7fd099d73640 1 -- 172.21.15.87:0/2963122862 learned_addr learned my addr 172.21.15.87:0/2963122862 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.014+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780af910 con 0x7f327004b4f0 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.014+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32780afa70 con 0x7f327004b4f0 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.014+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780afe20 con 0x7f327004b4f0 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.014+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32700c09e0 con 0x7f327004b4f0 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.014+0000 7fd099572640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0940c7e10 0x7fd09414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.015+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32700d0a40 con 0x7f327004b4f0 2024-01-20T08:31:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.015+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f327004bcf0 con 0x7f327004b4f0 2024-01-20T08:31:44.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f803ad20 con 0x7f5a04157b70 2024-01-20T08:31:44.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f803ae80 con 0x7f5a04157b70 2024-01-20T08:31:44.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8044630 con 0x7f5a04157b70 2024-01-20T08:31:44.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7ff8e54dd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a04127690 con 0x7f5a04157b70 2024-01-20T08:31:44.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a04071800 con 0x7f5a04157b70 2024-01-20T08:31:44.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.017+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a04160de0 con 0x7f5a04157b70 2024-01-20T08:31:44.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.018+0000 7f2302ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 0x7f23040703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.019+0000 7fd099d73640 1 -- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0940c7e10 msgr2=0x7fd09414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.019+0000 7fd099d73640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0940c7e10 0x7fd09414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.019+0000 7fd099d73640 1 -- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd09414f7b0 msgr2=0x7fd09414b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.019+0000 7fd099d73640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd09414f7b0 0x7fd09414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.019+0000 7fd099d73640 1 -- 172.21.15.87:0/2963122862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084009400 con 0x7fd094070d00 2024-01-20T08:31:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.032+0000 7f2302ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 0x7f23040703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36402/0 (socket says 172.21.15.87:36402) 2024-01-20T08:31:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.032+0000 7f2302ffd640 1 -- 172.21.15.87:0/3909462515 learned_addr learned my addr 172.21.15.87:0/3909462515 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.032+0000 7ff8e54dd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36396/0 (socket says 172.21.15.87:36396) 2024-01-20T08:31:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.032+0000 7ff8e54dd640 1 -- 172.21.15.87:0/3216406421 learned_addr learned my addr 172.21.15.87:0/3216406421 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.034+0000 7f73813bf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f737c0719d0 0x7f737c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.034+0000 7f73813bf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f737c0719d0 0x7f737c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36412/0 (socket says 172.21.15.87:36412) 2024-01-20T08:31:44.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.034+0000 7f73813bf640 1 -- 172.21.15.87:0/2189452308 learned_addr learned my addr 172.21.15.87:0/2189452308 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.036+0000 7f2302ffd640 1 -- 172.21.15.87:0/3909462515 >> v2:172.21.15.87:3300/0 conn(0x7f2304156a30 msgr2=0x7f2304156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.036+0000 7f2302ffd640 1 --2- 172.21.15.87:0/3909462515 >> v2:172.21.15.87:3300/0 conn(0x7f2304156a30 0x7f2304156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.036+0000 7f2302ffd640 1 -- 172.21.15.87:0/3909462515 >> v1:172.21.15.87:6789/0 conn(0x7f230406fff0 legacy=0x7f23040b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.037+0000 7f2302ffd640 1 -- 172.21.15.87:0/3909462515 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23040c3f00 con 0x7f23040c6810 2024-01-20T08:31:44.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.042+0000 7ff8e54dd640 1 -- 172.21.15.87:0/3216406421 >> v2:172.21.15.87:3300/0 conn(0x7ff8e015b740 msgr2=0x7ff8e00c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.042+0000 7ff8e54dd640 1 --2- 172.21.15.87:0/3216406421 >> v2:172.21.15.87:3300/0 conn(0x7ff8e015b740 0x7ff8e00c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.042+0000 7ff8e54dd640 1 -- 172.21.15.87:0/3216406421 >> v1:172.21.15.87:6789/0 conn(0x7ff8e00c6ca0 legacy=0x7ff8e0151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.042+0000 7ff8e54dd640 1 -- 172.21.15.87:0/3216406421 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e006ab90 con 0x7ff8e006b1c0 2024-01-20T08:31:44.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.042+0000 7f2302ffd640 1 --2- 172.21.15.87:0/3909462515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 0x7f23040703d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009870 tx=0x7f22f802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b9fb59296189a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.067+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f32780b6040 con 0x7f327004b4f0 2024-01-20T08:31:44.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.068+0000 7f3266ffd640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f325806af20 0x7f325806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.068+0000 7f327659c640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f325806af20 0x7f325806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.069+0000 7f327659c640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f325806af20 0x7f325806d3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f3268009060 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.073+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f32780f5020 con 0x7f327004b4f0 2024-01-20T08:31:44.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.073+0000 7f3266ffd640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f3258074520 0x7f32580769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.073+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 --> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3258077070 con 0x7f3258074520 2024-01-20T08:31:44.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.073+0000 7f3276d9d640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f3258074520 0x7f32580769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.073+0000 7f3276d9d640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f3258074520 0x7f32580769a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.074+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== osd.2 v2:172.21.15.87:6808/3102513365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3258077070 con 0x7f3258074520 2024-01-20T08:31:44.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.076+0000 7f73813bf640 1 -- 172.21.15.87:0/2189452308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 msgr2=0x7f737c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.076+0000 7f73813bf640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f737c157b70 0x7f737c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.076+0000 7f73813bf640 1 -- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f737c071200 msgr2=0x7f737c15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.076+0000 7f73813bf640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f737c071200 0x7f737c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.076+0000 7f73813bf640 1 -- 172.21.15.87:0/2189452308 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c009400 con 0x7f737c0719d0 2024-01-20T08:31:44.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.076+0000 7ff8e54dd640 1 --2- 172.21.15.87:0/3216406421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c7970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009870 tx=0x7ff8dc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659bc470148dc8d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.077+0000 7f9ee5a8a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ee006b1c0 0x7f9ee00c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.077+0000 7f9ee5a8a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ee006b1c0 0x7f9ee00c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50170/0 (socket says 172.21.15.87:50170) 2024-01-20T08:31:44.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.077+0000 7f9ee5a8a640 1 -- 172.21.15.87:0/1870945876 learned_addr learned my addr 172.21.15.87:0/1870945876 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.080+0000 7f9ee5289640 1 --2- 172.21.15.87:0/1870945876 >> v2:172.21.15.87:3300/0 conn(0x7f9ee015b740 0x7f9ee00c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.080+0000 7fd099d73640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd094070d00 0x7fd094148d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd09000b3e0 tx=0x7fd09000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.081+0000 7f23027fc640 1 -- 172.21.15.87:0/3909462515 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80317c0 con 0x7f23040c6810 2024-01-20T08:31:44.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.081+0000 7f23027fc640 1 -- 172.21.15.87:0/3909462515 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f8031920 con 0x7f23040c6810 2024-01-20T08:31:44.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.081+0000 7f23027fc640 1 -- 172.21.15.87:0/3909462515 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8031cb0 con 0x7f23040c6810 2024-01-20T08:31:44.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 -- 172.21.15.87:0/3909462515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 msgr2=0x7f23040703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 --2- 172.21.15.87:0/3909462515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 0x7f23040703d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009870 tx=0x7f22f802b860 comp rx=0 tx=0).stop 2024-01-20T08:31:44.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 -- 172.21.15.87:0/3909462515 shutdown_connections 2024-01-20T08:31:44.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 --2- 172.21.15.87:0/3909462515 >> v2:172.21.15.87:3300/0 conn(0x7f2304156a30 0x7f2304156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 --2- 172.21.15.87:0/3909462515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23040c6810 0x7f23040703d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 -- 172.21.15.87:0/3909462515 >> 172.21.15.87:0/3909462515 conn(0x7f230406b6c0 msgr2=0x7f230406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 -- 172.21.15.87:0/3909462515 shutdown_connections 2024-01-20T08:31:44.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.082+0000 7f230a246640 1 -- 172.21.15.87:0/3909462515 wait complete. 2024-01-20T08:31:44.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.084+0000 7f230a246640 1 Processor -- start 2024-01-20T08:31:44.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.084+0000 7f230a246640 1 -- start start 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f230a246640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f230406fff0 0x7f230415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f230a246640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23040c6810 0x7f2304147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f230a246640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2304156a30 0x7f23041517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f230a246640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304159ae0 con 0x7f230406fff0 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f230a246640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041522c0 con 0x7f23040c6810 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f73813bf640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f737c0719d0 0x7f737c15ad90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f737400efc0 tx=0x7f737400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.085+0000 7f230a246640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23041f1d00 con 0x7f2304156a30 2024-01-20T08:31:44.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.087+0000 7f9ee5289640 1 -- 172.21.15.87:0/1870945876 >> v2:172.21.15.116:3300/0 conn(0x7f9ee006b1c0 msgr2=0x7f9ee00c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.087+0000 7f9ee5289640 1 --2- 172.21.15.87:0/1870945876 >> v2:172.21.15.116:3300/0 conn(0x7f9ee006b1c0 0x7f9ee00c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.087+0000 7f9ee5289640 1 -- 172.21.15.87:0/1870945876 >> v1:172.21.15.87:6789/0 conn(0x7f9ee00c6ca0 legacy=0x7f9ee0151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.087+0000 7f9ee5289640 1 -- 172.21.15.87:0/1870945876 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee015ac60 con 0x7f9ee015b740 2024-01-20T08:31:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.090+0000 7f9ee5289640 1 --2- 172.21.15.87:0/1870945876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee015b740 0x7f9ee00c6760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0070b50 tx=0x7f9ed002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c845e37a279e8d2f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 --> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3270007420 con 0x7f3258074520 2024-01-20T08:31:44.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f3266ffd640 1 -- 172.21.15.87:0/2702638459 <== osd.2 v2:172.21.15.87:6808/3102513365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f326c001120 con 0x7f3258074520 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f3258074520 msgr2=0x7f32580769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f3258074520 0x7f32580769a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f325806af20 msgr2=0x7f325806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f325806af20 0x7f325806d3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f3268009060 comp rx=0 tx=0).stop 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b4f0 msgr2=0x7f32700d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f327cc9b640 1 --2- 172.21.15.87:0/2702638459 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f327004b4f0 0x7f32700d0fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3278067d30 tx=0x7f32780623e0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.092+0000 7f3276d9d640 1 -- 172.21.15.87:0/2702638459 reap_dead start 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.093+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 shutdown_connections 2024-01-20T08:31:44.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.093+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 >> 172.21.15.87:0/2702638459 conn(0x7f3270050170 msgr2=0x7f327004cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.093+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 shutdown_connections 2024-01-20T08:31:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.093+0000 7f327cc9b640 1 -- 172.21.15.87:0/2702638459 wait complete. 2024-01-20T08:31:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.094+0000 7f2302ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23040c6810 0x7f2304147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7f2302ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23040c6810 0x7f2304147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50196/0 (socket says 172.21.15.87:50196) 2024-01-20T08:31:44.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7f2302ffd640 1 -- 172.21.15.87:0/2552748291 learned_addr learned my addr 172.21.15.87:0/2552748291 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090011830 con 0x7fd094070d00 2024-01-20T08:31:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0900119b0 con 0x7fd094070d00 2024-01-20T08:31:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09414a0e0 con 0x7fd094070d00 2024-01-20T08:31:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7f866e7fc640 1 -- 172.21.15.87:0/3248405383 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866802cc60 con 0x7f86701612c0 2024-01-20T08:31:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090011d60 con 0x7fd094070d00 2024-01-20T08:31:44.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09414a6a0 con 0x7fd094070d00 2024-01-20T08:31:44.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.096+0000 7f866e7fc640 1 -- 172.21.15.87:0/3248405383 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866802cdc0 con 0x7f86701612c0 2024-01-20T08:31:44.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f866e7fc640 1 -- 172.21.15.87:0/3248405383 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668034660 con 0x7f86701612c0 2024-01-20T08:31:44.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 -- 172.21.15.87:0/3248405383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86701612c0 msgr2=0x7f86701685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 --2- 172.21.15.87:0/3248405383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86701612c0 0x7f86701685f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8668009d90 tx=0x7f866802beb0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd09414fd20 con 0x7fd094070d00 2024-01-20T08:31:44.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 -- 172.21.15.87:0/3248405383 shutdown_connections 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 --2- 172.21.15.87:0/3248405383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86701612c0 0x7f86701685f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 --2- 172.21.15.87:0/3248405383 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 0x7f867015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 -- 172.21.15.87:0/3248405383 >> 172.21.15.87:0/3248405383 conn(0x7f867006b6c0 msgr2=0x7f867006baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 -- 172.21.15.87:0/3248405383 shutdown_connections 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.097+0000 7f867544f640 1 -- 172.21.15.87:0/3248405383 wait complete. 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 Processor -- start 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 -- start start 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 0x7f8670160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86700c8c00 0x7f8670158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86701612c0 0x7f86701f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c1990 con 0x7f86700c8c00 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700bfcb0 con 0x7f86701612c0 2024-01-20T08:31:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.099+0000 7f867544f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86700bff90 con 0x7f86700c8820 2024-01-20T08:31:44.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.100+0000 7ff8d7fff640 1 -- 172.21.15.87:0/3216406421 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc030840 con 0x7ff8e006b1c0 2024-01-20T08:31:44.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.100+0000 7ff8d7fff640 1 -- 172.21.15.87:0/3216406421 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc0309a0 con 0x7ff8e006b1c0 2024-01-20T08:31:44.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.100+0000 7f23037fe640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f230406fff0 0x7f230415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.100+0000 7ff8d7fff640 1 -- 172.21.15.87:0/3216406421 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc030d30 con 0x7ff8e006b1c0 2024-01-20T08:31:44.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.100+0000 7f8667fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86700c8c00 0x7f8670158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 -- 172.21.15.87:0/3216406421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 msgr2=0x7ff8e00c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 --2- 172.21.15.87:0/3216406421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c7970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009870 tx=0x7ff8dc02b980 comp rx=0 tx=0).stop 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 -- 172.21.15.87:0/3216406421 shutdown_connections 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 --2- 172.21.15.87:0/3216406421 >> v2:172.21.15.87:3300/0 conn(0x7ff8e015b740 0x7ff8e00c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 --2- 172.21.15.87:0/3216406421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c7970 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 -- 172.21.15.87:0/3216406421 >> 172.21.15.87:0/3216406421 conn(0x7ff8e00723c0 msgr2=0x7ff8e006fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 -- 172.21.15.87:0/3216406421 shutdown_connections 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.101+0000 7ff8e7741640 1 -- 172.21.15.87:0/3216406421 wait complete. 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.102+0000 7ff8e7741640 1 Processor -- start 2024-01-20T08:31:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.102+0000 7ff8e7741640 1 -- start start 2024-01-20T08:31:44.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.103+0000 7f8667fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86700c8c00 0x7f8670158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37814/0 (socket says 172.21.15.87:37814) 2024-01-20T08:31:44.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.103+0000 7f8667fff640 1 -- 172.21.15.87:0/3238756197 learned_addr learned my addr 172.21.15.87:0/3238756197 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.103+0000 7ff8e7741640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.103+0000 7ff8e7741640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8e00c6ca0 0x7ff8e0157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.103+0000 7ff8e7741640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8e015b740 0x7ff8e01579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.104+0000 7ff8e7741640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00c0e80 con 0x7ff8e015b740 2024-01-20T08:31:44.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.104+0000 7ff8e7741640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00c2850 con 0x7ff8e00c6ca0 2024-01-20T08:31:44.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.104+0000 7ff8e7741640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00c2150 con 0x7ff8e006b1c0 2024-01-20T08:31:44.109 INFO:teuthology.orchestra.run.smithi087.stdout:60129542159 2024-01-20T08:31:44.109 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-20T08:31:44.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.106+0000 7f23037fe640 1 -- 172.21.15.87:0/2552748291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2304156a30 msgr2=0x7f23041517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.106+0000 7f23037fe640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2304156a30 0x7f23041517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.106+0000 7f23037fe640 1 -- 172.21.15.87:0/2552748291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23040c6810 msgr2=0x7f2304147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.106+0000 7f23037fe640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23040c6810 0x7f2304147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.106+0000 7f23037fe640 1 -- 172.21.15.87:0/2552748291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f80094d0 con 0x7f230406fff0 2024-01-20T08:31:44.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.109+0000 7f9ee4a88640 1 -- 172.21.15.87:0/1870945876 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0031830 con 0x7f9ee015b740 2024-01-20T08:31:44.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.109+0000 7f9ee4a88640 1 -- 172.21.15.87:0/1870945876 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0031990 con 0x7f9ee015b740 2024-01-20T08:31:44.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.109+0000 7f9ee4a88640 1 -- 172.21.15.87:0/1870945876 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0031d20 con 0x7f9ee015b740 2024-01-20T08:31:44.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1870945876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee015b740 msgr2=0x7f9ee00c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 --2- 172.21.15.87:0/1870945876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee015b740 0x7f9ee00c6760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0070b50 tx=0x7f9ed002b8d0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1870945876 shutdown_connections 2024-01-20T08:31:44.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 --2- 172.21.15.87:0/1870945876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee015b740 0x7f9ee00c6760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 --2- 172.21.15.87:0/1870945876 >> v2:172.21.15.116:3300/0 conn(0x7f9ee006b1c0 0x7f9ee00c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1870945876 >> 172.21.15.87:0/1870945876 conn(0x7f9ee00723c0 msgr2=0x7f9ee006fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1870945876 shutdown_connections 2024-01-20T08:31:44.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.110+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1870945876 wait complete. 2024-01-20T08:31:44.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.112+0000 7f9ee7cee640 1 Processor -- start 2024-01-20T08:31:44.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.112+0000 7f9ee7cee640 1 -- start start 2024-01-20T08:31:44.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.113+0000 7f9ee7cee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee006b1c0 0x7f9ee00ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.113+0000 7f9ee7cee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee00c6ca0 0x7f9ee01f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.113+0000 7f9ee7cee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee015b740 0x7f9ee0144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.113+0000 7f9ee7cee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01589a0 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.113+0000 7f9ee7cee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015a8a0 con 0x7f9ee015b740 2024-01-20T08:31:44.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.113+0000 7f9ee7cee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015a2f0 con 0x7f9ee006b1c0 2024-01-20T08:31:44.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.114+0000 7f8667fff640 1 -- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 msgr2=0x7f8670160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.114+0000 7f8667fff640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86700c8820 0x7f8670160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.114+0000 7f8667fff640 1 -- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86701612c0 msgr2=0x7f86701f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.114+0000 7f8667fff640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86701612c0 0x7f86701f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.114+0000 7f8667fff640 1 -- 172.21.15.87:0/3238756197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8668009a70 con 0x7f86700c8c00 2024-01-20T08:31:44.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.115+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f59f8044790 con 0x7f5a04157b70 2024-01-20T08:31:44.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.115+0000 7f59f2ffd640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f59e406af50 0x7f59e406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.115+0000 7f5a0259c640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f59e406af50 0x7f59e406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.116+0000 7f5a0259c640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f59e406af50 0x7f59e406d410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59f4000f60 tx=0x7f59f40079b0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.116+0000 7f23037fe640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f230406fff0 0x7f230415a1e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c410 tx=0x7f22f400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.118+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f59f807b8a0 con 0x7f5a04157b70 2024-01-20T08:31:44.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.118+0000 7f59f2ffd640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f59e4074550 0x7f59e40769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.118+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 --> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59e40770a0 con 0x7f59e4074550 2024-01-20T08:31:44.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.118+0000 7f5a02d9d640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f59e4074550 0x7f59e40769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.119+0000 7f5a02d9d640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f59e4074550 0x7f59e40769d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.120+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== osd.7 v2:172.21.15.116:6800/1436327026 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f59e40770a0 con 0x7f59e4074550 2024-01-20T08:31:44.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.120+0000 7f8667fff640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86700c8c00 0x7f8670158470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f866000c4a0 tx=0x7f866000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.120+0000 7f9ee5289640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee00c6ca0 0x7f9ee01f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.121+0000 7ff8e5cde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8e015b740 0x7ff8e01579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.121+0000 7f9ee5289640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee00c6ca0 0x7f9ee01f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37828/0 (socket says 172.21.15.87:37828) 2024-01-20T08:31:44.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.121+0000 7f9ee5289640 1 -- 172.21.15.87:0/1859665391 learned_addr learned my addr 172.21.15.87:0/1859665391 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.125+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400cd40 con 0x7f737c0719d0 2024-01-20T08:31:44.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.125+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737400f040 con 0x7f737c0719d0 2024-01-20T08:31:44.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.125+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c158930 con 0x7f737c0719d0 2024-01-20T08:31:44.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400cea0 con 0x7f737c0719d0 2024-01-20T08:31:44.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f9ee5289640 1 -- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee006b1c0 msgr2=0x7f9ee00ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f9ee5289640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee006b1c0 0x7f9ee00ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f9ee5289640 1 -- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee015b740 msgr2=0x7f9ee0144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f9ee5289640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee015b740 0x7f9ee0144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f9ee5289640 1 -- 172.21.15.87:0/1859665391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009400 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c158cc0 con 0x7f737c0719d0 2024-01-20T08:31:44.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7ff8e5cde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8e015b740 0x7ff8e01579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37818/0 (socket says 172.21.15.87:37818) 2024-01-20T08:31:44.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.126+0000 7ff8e5cde640 1 -- 172.21.15.87:0/3114909779 learned_addr learned my addr 172.21.15.87:0/3114909779 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.127+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f737c0716f0 con 0x7f737c0719d0 2024-01-20T08:31:44.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.127+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd09001a400 con 0x7fd094070d00 2024-01-20T08:31:44.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.127+0000 7fd08a7fc640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd07806af20 0x7fd07806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.127+0000 7fd099572640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd07806af20 0x7fd07806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.128+0000 7fd099572640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd07806af20 0x7fd07806d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd0941550d0 tx=0x7fd08c0061f0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.129+0000 7ff8e54dd640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.129+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40035f0 con 0x7f230406fff0 2024-01-20T08:31:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.129+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4003770 con 0x7f230406fff0 2024-01-20T08:31:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.129+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4003b20 con 0x7f230406fff0 2024-01-20T08:31:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.129+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2304159d50 con 0x7f230406fff0 2024-01-20T08:31:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.129+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2304151120 con 0x7f230406fff0 2024-01-20T08:31:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.130+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23040c8820 con 0x7f230406fff0 2024-01-20T08:31:44.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.131+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fd090058f30 con 0x7fd094070d00 2024-01-20T08:31:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.132+0000 7fd08a7fc640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7fd078074520 0x7fd0780769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.132+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 --> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd078077070 con 0x7fd078074520 2024-01-20T08:31:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.132+0000 7fd09a574640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7fd078074520 0x7fd0780769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.132+0000 7fd09a574640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7fd078074520 0x7fd0780769a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.133+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd090009210 con 0x7fd094070d00 2024-01-20T08:31:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.133+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== osd.5 v2:172.21.15.116:6816/4162650663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd078077070 con 0x7fd078074520 2024-01-20T08:31:44.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.134+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660005ce0 con 0x7f86700c8c00 2024-01-20T08:31:44.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.134+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660005e40 con 0x7f86700c8c00 2024-01-20T08:31:44.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.134+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660007270 con 0x7f86700c8c00 2024-01-20T08:31:44.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.134+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86700c0270 con 0x7f86700c8c00 2024-01-20T08:31:44.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.134+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86700c1f30 con 0x7f86700c8c00 2024-01-20T08:31:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.135+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8670074510 con 0x7f86700c8c00 2024-01-20T08:31:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.135+0000 7ff8e4cdc640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8e00c6ca0 0x7ff8e0157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.136+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 --> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5a040c9f80 con 0x7f59e4074550 2024-01-20T08:31:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.136+0000 7f59f2ffd640 1 -- 172.21.15.87:0/2031402954 <== osd.7 v2:172.21.15.116:6800/1436327026 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f59fc001fe0 con 0x7f59e4074550 2024-01-20T08:31:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f59e4074550 msgr2=0x7f59e40769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f59e4074550 0x7f59e40769d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f59e406af50 msgr2=0x7f59e406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f59e406af50 0x7f59e406d410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59f4000f60 tx=0x7f59f40079b0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a04157b70 msgr2=0x7f5a04156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 --2- 172.21.15.87:0/2031402954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5a04157b70 0x7f5a04156060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59f8033000 tx=0x7f59f80046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a02d9d640 1 -- 172.21.15.87:0/2031402954 reap_dead start 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 shutdown_connections 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 >> 172.21.15.87:0/2031402954 conn(0x7f5a040ca970 msgr2=0x7f5a040c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 shutdown_connections 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7f5a0359e640 1 -- 172.21.15.87:0/2031402954 wait complete. 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7ff8e5cde640 1 -- 172.21.15.87:0/3114909779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 msgr2=0x7ff8e00c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7ff8e5cde640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8e006b1c0 0x7ff8e00c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7ff8e5cde640 1 -- 172.21.15.87:0/3114909779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8e00c6ca0 msgr2=0x7ff8e0157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7ff8e5cde640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8e00c6ca0 0x7ff8e0157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.137+0000 7ff8e5cde640 1 -- 172.21.15.87:0/3114909779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0094d0 con 0x7ff8e015b740 2024-01-20T08:31:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.140+0000 7f9ee5289640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee00c6ca0 0x7f9ee01f2860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.152 INFO:teuthology.orchestra.run.smithi087.stdout:55834574862 2024-01-20T08:31:44.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-20T08:31:44.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.150+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 --> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0940c71e0 con 0x7fd078074520 2024-01-20T08:31:44.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.150+0000 7fd08a7fc640 1 -- 172.21.15.87:0/2963122862 <== osd.5 v2:172.21.15.116:6816/4162650663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd084030ce0 con 0x7fd078074520 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7ff8e5cde640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8e015b740 0x7ff8e01579d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c7b0 tx=0x7ff8d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7fd078074520 msgr2=0x7fd0780769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7fd078074520 0x7fd0780769a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd07806af20 msgr2=0x7fd07806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd07806af20 0x7fd07806d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd0941550d0 tx=0x7fd08c0061f0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd094070d00 msgr2=0x7fd094148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 --2- 172.21.15.87:0/2963122862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd094070d00 0x7fd094148d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd09000b3e0 tx=0x7fd09000b8a0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09a574640 1 -- 172.21.15.87:0/2963122862 reap_dead start 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 shutdown_connections 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 >> 172.21.15.87:0/2963122862 conn(0x7fd09406e8c0 msgr2=0x7fd09406eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 shutdown_connections 2024-01-20T08:31:44.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.151+0000 7fd09bfd7640 1 -- 172.21.15.87:0/2963122862 wait complete. 2024-01-20T08:31:44.165 INFO:teuthology.orchestra.run.smithi087.stdout:60129542159 2024-01-20T08:31:44.166 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-20T08:31:44.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.168+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8660007490 con 0x7f86700c8c00 2024-01-20T08:31:44.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.168+0000 7f86677fe640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f865006af20 0x7f865006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.168+0000 7f866effd640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f865006af20 0x7f865006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.169+0000 7f866effd640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f865006af20 0x7f865006d3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8658009710 tx=0x7f8658006c90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.173+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed003a830 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.173+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed003a9b0 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.173+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed003ad60 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.173+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee00ca540 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.173+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee00caab0 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.174+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ee00c7eb0 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f8660057f10 con 0x7f86700c8c00 2024-01-20T08:31:44.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7f86677fe640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f86500745a0 0x7f8650076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 --> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86500770f0 con 0x7f86500745a0 2024-01-20T08:31:44.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8660001200 con 0x7f86700c8c00 2024-01-20T08:31:44.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7f866f7fe640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f86500745a0 0x7f8650076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8003d80 con 0x7ff8e015b740 2024-01-20T08:31:44.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8005070 con 0x7ff8e015b740 2024-01-20T08:31:44.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8005420 con 0x7ff8e015b740 2024-01-20T08:31:44.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.185+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e00c10b0 con 0x7ff8e015b740 2024-01-20T08:31:44.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.186+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e00c2da0 con 0x7ff8e015b740 2024-01-20T08:31:44.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.186+0000 7f866f7fe640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f86500745a0 0x7f8650076a20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.187+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8e00c7eb0 con 0x7ff8e015b740 2024-01-20T08:31:44.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.187+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== osd.6 v2:172.21.15.116:6821/3458193509 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f86500770f0 con 0x7f86500745a0 2024-01-20T08:31:44.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.213+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9ed004d450 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.214+0000 7f9ed67fc640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9ec406af50 0x7f9ec406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.214+0000 7f9ee5a8a640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9ec406af50 0x7f9ec406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.215+0000 7f9ee5a8a640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9ec406af50 0x7f9ec406d410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0094a0 tx=0x7f9edc0093f0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.217+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 --> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8670074510 con 0x7f86500745a0 2024-01-20T08:31:44.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f86677fe640 1 -- 172.21.15.87:0/3238756197 <== osd.6 v2:172.21.15.116:6821/3458193509 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8670074510 con 0x7f86500745a0 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f86500745a0 msgr2=0x7f8650076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f867544f640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f86500745a0 0x7f8650076a20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f865006af20 msgr2=0x7f865006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f867544f640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f865006af20 0x7f865006d3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8658009710 tx=0x7f8658006c90 comp rx=0 tx=0).stop 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86700c8c00 msgr2=0x7f8670158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.218+0000 7f867544f640 1 --2- 172.21.15.87:0/3238756197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86700c8c00 0x7f8670158470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f866000c4a0 tx=0x7f866000c960 comp rx=0 tx=0).stop 2024-01-20T08:31:44.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.219+0000 7f866f7fe640 1 -- 172.21.15.87:0/3238756197 reap_dead start 2024-01-20T08:31:44.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.219+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 shutdown_connections 2024-01-20T08:31:44.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.219+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 >> 172.21.15.87:0/3238756197 conn(0x7f867006b6c0 msgr2=0x7f86700c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.219+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 shutdown_connections 2024-01-20T08:31:44.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.219+0000 7f867544f640 1 -- 172.21.15.87:0/3238756197 wait complete. 2024-01-20T08:31:44.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.222+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7374014680 con 0x7f737c0719d0 2024-01-20T08:31:44.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.222+0000 7f73727fc640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f736006af50 0x7f736006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.222+0000 7f7381bc0640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f736006af50 0x7f736006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.223+0000 7f7381bc0640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f736006af50 0x7f736006d410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f736c036000 tx=0x7f736c030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.224+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9ed007b0c0 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.225+0000 7f9ed67fc640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9ec40745d0 0x7f9ec4076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.225+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 --> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ec4077120 con 0x7f9ec40745d0 2024-01-20T08:31:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.225+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ed0076d00 con 0x7f9ee00c6ca0 2024-01-20T08:31:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.225+0000 7f9ee628b640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9ec40745d0 0x7f9ec4076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.225+0000 7f9ee628b640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9ec40745d0 0x7f9ec4076a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.226+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7374059a40 con 0x7f737c0719d0 2024-01-20T08:31:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.226+0000 7f73727fc640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7f7360074550 0x7f73600769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.226+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 --> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73600770a0 con 0x7f7360074550 2024-01-20T08:31:44.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.227+0000 7f73823c1640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7f7360074550 0x7f73600769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.227+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== osd.4 v2:172.21.15.116:6808/2305241895 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9ec4077120 con 0x7f9ec40745d0 2024-01-20T08:31:44.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.227+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7374055680 con 0x7f737c0719d0 2024-01-20T08:31:44.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.227+0000 7f73823c1640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7f7360074550 0x7f73600769d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.228+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== osd.3 v2:172.21.15.87:6816/973971716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f73600770a0 con 0x7f7360074550 2024-01-20T08:31:44.239 INFO:teuthology.orchestra.run.smithi087.stdout:64424509457 2024-01-20T08:31:44.239 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-20T08:31:44.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.244+0000 7f331a9bc640 1 Processor -- start 2024-01-20T08:31:44.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.244+0000 7f331a9bc640 1 -- start start 2024-01-20T08:31:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.245+0000 7f331a9bc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f33140cc170 0x7f331406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.246+0000 7f331a9bc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f33140703c0 0x7f3314159610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.246+0000 7f331a9bc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33140c2df0 con 0x7f33140cbd90 2024-01-20T08:31:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.246+0000 7f33199ba640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f33140cbd90 0x7f3314153420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:38996/0 (socket says 172.21.15.87:38996) 2024-01-20T08:31:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.246+0000 7f33199ba640 1 -- 172.21.15.87:0/2950720608 learned_addr learned my addr 172.21.15.87:0/2950720608 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.246+0000 7f331a9bc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331415d8b0 con 0x7f33140703c0 2024-01-20T08:31:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.246+0000 7f331a9bc640 1 -- 172.21.15.87:0/2950720608 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331406f9e0 con 0x7f33140cc170 2024-01-20T08:31:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.248+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 --> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ee015b220 con 0x7f9ec40745d0 2024-01-20T08:31:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.248+0000 7f33191b9640 1 --2- 172.21.15.87:0/2950720608 >> v2:172.21.15.116:3300/0 conn(0x7f33140cc170 0x7f331406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.249+0000 7f33189b8640 1 -- 172.21.15.87:0/2950720608 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574170193 0 0) 0x7f3310001110 con 0x7f33140cbd90 2024-01-20T08:31:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.249+0000 7f33189b8640 1 -- 172.21.15.87:0/2950720608 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3300003520 con 0x7f33140cbd90 2024-01-20T08:31:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.249+0000 7f9ed67fc640 1 -- 172.21.15.87:0/1859665391 <== osd.4 v2:172.21.15.116:6808/2305241895 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ed8006b10 con 0x7f9ec40745d0 2024-01-20T08:31:44.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.249+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9ec40745d0 msgr2=0x7f9ec4076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.249+0000 7f9ee7cee640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9ec40745d0 0x7f9ec4076a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.249+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9ec406af50 msgr2=0x7f9ec406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.250+0000 7f9ee7cee640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9ec406af50 0x7f9ec406d410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0094a0 tx=0x7f9edc0093f0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.250+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee00c6ca0 msgr2=0x7f9ee01f2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.250+0000 7f9ee7cee640 1 --2- 172.21.15.87:0/1859665391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ee00c6ca0 0x7f9ee01f2860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed00046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.250+0000 7f9ee628b640 1 -- 172.21.15.87:0/1859665391 reap_dead start 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.250+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 shutdown_connections 2024-01-20T08:31:44.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.250+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 >> 172.21.15.87:0/1859665391 conn(0x7f9ee00723c0 msgr2=0x7f9ee006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.251+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f22f4003c80 con 0x7f230406fff0 2024-01-20T08:31:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.251+0000 7f22ebfff640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f22f006af20 0x7f22f006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.251+0000 7f2302ffd640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f22f006af20 0x7f22f006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.252+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 shutdown_connections 2024-01-20T08:31:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.252+0000 7f9ee7cee640 1 -- 172.21.15.87:0/1859665391 wait complete. 2024-01-20T08:31:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.252+0000 7f2302ffd640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f22f006af20 0x7f22f006d3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22f80090b0 tx=0x7f22f803d040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.255+0000 7f33191b9640 1 -- 172.21.15.87:0/2950720608 >> v2:172.21.15.87:3300/0 conn(0x7f33140703c0 msgr2=0x7f3314159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.255+0000 7f33191b9640 1 --2- 172.21.15.87:0/2950720608 >> v2:172.21.15.87:3300/0 conn(0x7f33140703c0 0x7f3314159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.255+0000 7f33191b9640 1 -- 172.21.15.87:0/2950720608 >> v1:172.21.15.116:6789/0 conn(0x7f33140cbd90 legacy=0x7f3314153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.255+0000 7f33191b9640 1 -- 172.21.15.87:0/2950720608 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331415cca0 con 0x7f33140cc170 2024-01-20T08:31:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.255+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 --> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f737c072650 con 0x7f7360074550 2024-01-20T08:31:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.255+0000 7f73727fc640 1 -- 172.21.15.87:0/2189452308 <== osd.3 v2:172.21.15.87:6816/973971716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f73780038d0 con 0x7f7360074550 2024-01-20T08:31:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.256+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7f7360074550 msgr2=0x7f73600769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.256+0000 7f7382bc2640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7f7360074550 0x7f73600769d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.256+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f736006af50 msgr2=0x7f736006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.256+0000 7f7382bc2640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f736006af50 0x7f736006d410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f736c036000 tx=0x7f736c030000 comp rx=0 tx=0).stop 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.256+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f737c0719d0 msgr2=0x7f737c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.256+0000 7f7382bc2640 1 --2- 172.21.15.87:0/2189452308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f737c0719d0 0x7f737c15ad90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f737400efc0 tx=0x7f737400c490 comp rx=0 tx=0).stop 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.257+0000 7f73823c1640 1 -- 172.21.15.87:0/2189452308 reap_dead start 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.257+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 shutdown_connections 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.257+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 >> 172.21.15.87:0/2189452308 conn(0x7f737c0ca970 msgr2=0x7f737c0c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.257+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 shutdown_connections 2024-01-20T08:31:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.257+0000 7f7382bc2640 1 -- 172.21.15.87:0/2189452308 wait complete. 2024-01-20T08:31:44.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.257+0000 7f33191b9640 1 --2- 172.21.15.87:0/2950720608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cc170 0x7f331406ae20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f33140ccc20 tx=0x7f330402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e40068cc1fea89 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.261+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f22f4057fc0 con 0x7f230406fff0 2024-01-20T08:31:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.261+0000 7f22ebfff640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7f22f00745a0 0x7f22f0076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.261+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 --> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22f00770f0 con 0x7f22f00745a0 2024-01-20T08:31:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.261+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22f4054300 con 0x7f230406fff0 2024-01-20T08:31:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.261+0000 7f2303fff640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7f22f00745a0 0x7f22f0076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.262+0000 7f33189b8640 1 -- 172.21.15.87:0/2950720608 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304031640 con 0x7f33140cc170 2024-01-20T08:31:44.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.262+0000 7f33189b8640 1 -- 172.21.15.87:0/2950720608 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33040317a0 con 0x7f33140cc170 2024-01-20T08:31:44.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.262+0000 7f33189b8640 1 -- 172.21.15.87:0/2950720608 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304031b70 con 0x7f33140cc170 2024-01-20T08:31:44.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.262+0000 7f2303fff640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7f22f00745a0 0x7f22f0076a20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 -- 172.21.15.87:0/2950720608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cc170 msgr2=0x7f331406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 --2- 172.21.15.87:0/2950720608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cc170 0x7f331406ae20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f33140ccc20 tx=0x7f330402b8f0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 -- 172.21.15.87:0/2950720608 shutdown_connections 2024-01-20T08:31:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 --2- 172.21.15.87:0/2950720608 >> v2:172.21.15.87:3300/0 conn(0x7f33140703c0 0x7f3314159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 --2- 172.21.15.87:0/2950720608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cc170 0x7f331406ae20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 -- 172.21.15.87:0/2950720608 >> 172.21.15.87:0/2950720608 conn(0x7f3314071e50 msgr2=0x7f3314072260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== osd.1 v2:172.21.15.87:6818/2572242901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f22f00770f0 con 0x7f22f00745a0 2024-01-20T08:31:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.263+0000 7f331a9bc640 1 -- 172.21.15.87:0/2950720608 shutdown_connections 2024-01-20T08:31:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.264+0000 7f331a9bc640 1 -- 172.21.15.87:0/2950720608 wait complete. 2024-01-20T08:31:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.266+0000 7f331a9bc640 1 Processor -- start 2024-01-20T08:31:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.266+0000 7f331a9bc640 1 -- start start 2024-01-20T08:31:44.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.267+0000 7f331a9bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33140703c0 0x7f331414b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.268+0000 7f331a9bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 0x7f33140c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.268+0000 7f331a9bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33140cc170 0x7f33140c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.268+0000 7f331a9bc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141efb40 con 0x7f33140703c0 2024-01-20T08:31:44.272 INFO:teuthology.orchestra.run.smithi087.stdout:60129542158 2024-01-20T08:31:44.272 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-20T08:31:44.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.268+0000 7f331a9bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331414f8e0 con 0x7f33140cbd90 2024-01-20T08:31:44.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.268+0000 7f331a9bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331414fb10 con 0x7f33140cc170 2024-01-20T08:31:44.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.268+0000 7f33191b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 0x7f33140c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.269+0000 7f33191b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 0x7f33140c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50248/0 (socket says 172.21.15.87:50248) 2024-01-20T08:31:44.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.269+0000 7f33191b9640 1 -- 172.21.15.87:0/915100945 learned_addr learned my addr 172.21.15.87:0/915100945 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.269+0000 7f331a1bb640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33140cc170 0x7f33140c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.272+0000 7f33191b9640 1 -- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33140cc170 msgr2=0x7f33140c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.272+0000 7f33191b9640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33140cc170 0x7f33140c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.272+0000 7f33191b9640 1 -- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33140703c0 msgr2=0x7f331414b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.272+0000 7f33191b9640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33140703c0 0x7f331414b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.272+0000 7f33191b9640 1 -- 172.21.15.87:0/915100945 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304009400 con 0x7f33140cbd90 2024-01-20T08:31:44.278 INFO:teuthology.orchestra.run.smithi087.stdout:55834574862 2024-01-20T08:31:44.278 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-20T08:31:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.275+0000 7f33191b9640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 0x7f33140c44f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f330402bf00 tx=0x7f33040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.279+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330403cc50 con 0x7f33140cbd90 2024-01-20T08:31:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.279+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330403cdb0 con 0x7f33140cbd90 2024-01-20T08:31:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.279+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330403a690 con 0x7f33140cbd90 2024-01-20T08:31:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.279+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33141ef1a0 con 0x7f33140cbd90 2024-01-20T08:31:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.279+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141ef760 con 0x7f33140cbd90 2024-01-20T08:31:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.281+0000 7f6aee035640 1 Processor -- start 2024-01-20T08:31:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.282+0000 7f6aee035640 1 -- start start 2024-01-20T08:31:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.282+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33140ca200 con 0x7f33140cbd90 2024-01-20T08:31:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.283+0000 7f6aee035640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6ae00d9030 0x7f6ae004b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.283+0000 7f6aee035640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae00e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.283+0000 7f6aee035640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae00d9410 con 0x7f6ae00e27e0 2024-01-20T08:31:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.283+0000 7f6aee035640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae00df8b0 con 0x7f6ae00d9030 2024-01-20T08:31:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.283+0000 7f6aee035640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00461f0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.284+0000 7f6aec832640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae00e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.285+0000 7f6aec832640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae00e22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36474/0 (socket says 172.21.15.87:36474) 2024-01-20T08:31:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.285+0000 7f6aec832640 1 -- 172.21.15.87:0/602999669 learned_addr learned my addr 172.21.15.87:0/602999669 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.287+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 --> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f23040c3660 con 0x7f22f00745a0 2024-01-20T08:31:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.288+0000 7f22ebfff640 1 -- 172.21.15.87:0/2552748291 <== osd.1 v2:172.21.15.87:6818/2572242901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f22fc00a720 con 0x7f22f00745a0 2024-01-20T08:31:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.287+0000 7f6aec832640 1 -- 172.21.15.87:0/602999669 >> v2:172.21.15.87:3300/0 conn(0x7f6ae00d9030 msgr2=0x7f6ae004b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.287+0000 7f6aec832640 1 --2- 172.21.15.87:0/602999669 >> v2:172.21.15.87:3300/0 conn(0x7f6ae00d9030 0x7f6ae004b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.287+0000 7f6aec832640 1 -- 172.21.15.87:0/602999669 >> v1:172.21.15.116:6789/0 conn(0x7f6ae00e27e0 legacy=0x7f6ae00decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.288+0000 7f6aec832640 1 -- 172.21.15.87:0/602999669 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae00464d0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.289+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7f22f00745a0 msgr2=0x7f22f0076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.289+0000 7f230a246640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7f22f00745a0 0x7f22f0076a20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.289+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f22f006af20 msgr2=0x7f22f006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.289+0000 7f230a246640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f22f006af20 0x7f22f006d3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22f80090b0 tx=0x7f22f803d040 comp rx=0 tx=0).stop 2024-01-20T08:31:44.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.290+0000 7f6aec832640 1 --2- 172.21.15.87:0/602999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae00e22a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009870 tx=0x7f6ae402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8bf83eca7eb13b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.289+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f230406fff0 msgr2=0x7f230415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.289+0000 7f230a246640 1 --2- 172.21.15.87:0/2552748291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f230406fff0 0x7f230415a1e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c410 tx=0x7f22f400c810 comp rx=0 tx=0).stop 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.290+0000 7f2303fff640 1 -- 172.21.15.87:0/2552748291 reap_dead start 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.290+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 shutdown_connections 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.290+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 >> 172.21.15.87:0/2552748291 conn(0x7f230406b6c0 msgr2=0x7f23040cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.291+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 shutdown_connections 2024-01-20T08:31:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.291+0000 7f230a246640 1 -- 172.21.15.87:0/2552748291 wait complete. 2024-01-20T08:31:44.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.294+0000 7f6adffff640 1 -- 172.21.15.87:0/602999669 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4031870 con 0x7f6ae00e2da0 2024-01-20T08:31:44.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.294+0000 7f6adffff640 1 -- 172.21.15.87:0/602999669 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae40319d0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.294+0000 7f6adffff640 1 -- 172.21.15.87:0/602999669 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4031d60 con 0x7f6ae00e2da0 2024-01-20T08:31:44.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.295+0000 7f6aee035640 1 -- 172.21.15.87:0/602999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 msgr2=0x7f6ae00e22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.295+0000 7f6aee035640 1 --2- 172.21.15.87:0/602999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae00e22a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009870 tx=0x7f6ae402b910 comp rx=0 tx=0).stop 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.296+0000 7f6aee035640 1 -- 172.21.15.87:0/602999669 shutdown_connections 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.296+0000 7f6aee035640 1 --2- 172.21.15.87:0/602999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae00e22a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.296+0000 7f6aee035640 1 --2- 172.21.15.87:0/602999669 >> v2:172.21.15.87:3300/0 conn(0x7f6ae00d9030 0x7f6ae004b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.296+0000 7f6aee035640 1 -- 172.21.15.87:0/602999669 >> 172.21.15.87:0/602999669 conn(0x7f6ae004d4b0 msgr2=0x7f6ae004d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.296+0000 7f6aee035640 1 -- 172.21.15.87:0/602999669 shutdown_connections 2024-01-20T08:31:44.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.296+0000 7f6aee035640 1 -- 172.21.15.87:0/602999669 wait complete. 2024-01-20T08:31:44.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.298+0000 7f6aee035640 1 Processor -- start 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.298+0000 7f6aee035640 1 -- start start 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.299+0000 7f6aee035640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ae00d9030 0x7f6ae0036200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.299+0000 7f6aee035640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ae00e27e0 0x7f6ae01789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.300+0000 7f6aee035640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae0175450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.300+0000 7f6aee035640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00d4670 con 0x7f6ae00d9030 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.300+0000 7f6aee035640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0175030 con 0x7f6ae00e27e0 2024-01-20T08:31:44.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.300+0000 7f6aee035640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0174a40 con 0x7f6ae00e2da0 2024-01-20T08:31:44.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.301+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f330402f050 con 0x7f33140cbd90 2024-01-20T08:31:44.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.301+0000 7f330a7fc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f32f806aed0 0x7f32f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.302+0000 7f6aed834640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae0175450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.302+0000 7f6aed834640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae0175450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36478/0 (socket says 172.21.15.87:36478) 2024-01-20T08:31:44.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.302+0000 7f6aed834640 1 -- 172.21.15.87:0/907608340 learned_addr learned my addr 172.21.15.87:0/907608340 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.302+0000 7f33199ba640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f32f806aed0 0x7f32f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.302+0000 7f640359e640 1 Processor -- start 2024-01-20T08:31:44.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.303+0000 7f33199ba640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f32f806aed0 0x7f32f806d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3310005880 tx=0x7f3310001150 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.303+0000 7f640359e640 1 -- start start 2024-01-20T08:31:44.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.304+0000 7f640359e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f64040c97d0 0x7f64040725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.304+0000 7f640359e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640406ab40 con 0x7f64040c9bb0 2024-01-20T08:31:44.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.304+0000 7f640359e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640406ae20 con 0x7f640406efe0 2024-01-20T08:31:44.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.304+0000 7f640359e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64040c4ea0 con 0x7f64040c97d0 2024-01-20T08:31:44.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.306+0000 7f6aed834640 1 -- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ae00e27e0 msgr2=0x7f6ae01789c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.306+0000 7f6aed834640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ae00e27e0 0x7f6ae01789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.306+0000 7f6aed834640 1 -- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ae00d9030 msgr2=0x7f6ae0036200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.306+0000 7f6aed834640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ae00d9030 0x7f6ae0036200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.306+0000 7f6aed834640 1 -- 172.21.15.87:0/907608340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40094d0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.310 INFO:teuthology.orchestra.run.smithi087.stdout:60129542160 2024-01-20T08:31:44.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-20T08:31:44.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.308+0000 7f6aed834640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae0175450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c390 tx=0x7f6ad800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.312+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80035f0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.312+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8003770 con 0x7f6ae00e2da0 2024-01-20T08:31:44.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.313+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae00354f0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.313+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae0178f00 con 0x7f6ae00e2da0 2024-01-20T08:31:44.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.313+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8003b20 con 0x7f6ae00e2da0 2024-01-20T08:31:44.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.315+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae00057e0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.332+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6ad8003c80 con 0x7f6ae00e2da0 2024-01-20T08:31:44.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.332+0000 7f6addffb640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6acc06af00 0x7f6acc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.332+0000 7f6aed033640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6acc06af00 0x7f6acc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.333+0000 7f6aed033640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6acc06af00 0x7f6acc06d3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8064100 tx=0x7f6ae80673c0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.353+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff8d8005580 con 0x7ff8e015b740 2024-01-20T08:31:44.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.354+0000 7ff8d5ffb640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff8c406af00 0x7ff8c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.354+0000 7ff8e54dd640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff8c406af00 0x7ff8c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.354+0000 7ff8e54dd640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff8c406af00 0x7ff8c406d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc036000 tx=0x7ff8dc008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.364+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7ff8d8057950 con 0x7ff8e015b740 2024-01-20T08:31:44.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.364+0000 7ff8d5ffb640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7ff8c4074500 0x7ff8c4076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.364+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 --> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8c4077050 con 0x7ff8c4074500 2024-01-20T08:31:44.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.364+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8d8005830 con 0x7ff8e015b740 2024-01-20T08:31:44.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.364+0000 7ff8e4cdc640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7ff8c4074500 0x7ff8c4076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.365+0000 7ff8e4cdc640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7ff8c4074500 0x7ff8c4076980 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.366+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== osd.0 v2:172.21.15.87:6800/1449648137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff8c4077050 con 0x7ff8c4074500 2024-01-20T08:31:44.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.373+0000 7f6401d9b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f64040c9bb0 0x7f6404155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34190/0 (socket says 172.21.15.87:34190) 2024-01-20T08:31:44.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.373+0000 7f6401d9b640 1 -- 172.21.15.87:0/765044654 learned_addr learned my addr 172.21.15.87:0/765044654 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:44.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.380+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535347858 0 0) 0x7f640406ab40 con 0x7f64040c9bb0 2024-01-20T08:31:44.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.380+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ec003520 con 0x7f64040c9bb0 2024-01-20T08:31:44.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.382+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 --> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8e015af40 con 0x7ff8c4074500 2024-01-20T08:31:44.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.383+0000 7ff8d5ffb640 1 -- 172.21.15.87:0/3114909779 <== osd.0 v2:172.21.15.87:6800/1449648137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8d0009530 con 0x7ff8c4074500 2024-01-20T08:31:44.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.383+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7ff8c4074500 msgr2=0x7ff8c4076980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.383+0000 7ff8e7741640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7ff8c4074500 0x7ff8c4076980 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.383+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff8c406af00 msgr2=0x7ff8c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff8c406af00 0x7ff8c406d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc036000 tx=0x7ff8dc008030 comp rx=0 tx=0).stop 2024-01-20T08:31:44.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8e015b740 msgr2=0x7ff8e01579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 --2- 172.21.15.87:0/3114909779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8e015b740 0x7ff8e01579d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c7b0 tx=0x7ff8d800cc70 comp rx=0 tx=0).stop 2024-01-20T08:31:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e5cde640 1 -- 172.21.15.87:0/3114909779 reap_dead start 2024-01-20T08:31:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 shutdown_connections 2024-01-20T08:31:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 >> 172.21.15.87:0/3114909779 conn(0x7ff8e00723c0 msgr2=0x7ff8e0150f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 shutdown_connections 2024-01-20T08:31:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.384+0000 7ff8e7741640 1 -- 172.21.15.87:0/3114909779 wait complete. 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2817465688 0 0) 0x7f63ec003520 con 0x7f64040c9bb0 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 >> v2:172.21.15.116:3300/0 conn(0x7f64040c97d0 msgr2=0x7f64040725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 --2- 172.21.15.87:0/765044654 >> v2:172.21.15.116:3300/0 conn(0x7f64040c97d0 0x7f64040725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 >> v1:172.21.15.116:6789/0 conn(0x7f640406efe0 legacy=0x7f6404160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040c4500 con 0x7f64040c9bb0 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4017552009 0 0) 0x7f63f4001670 con 0x7f64040c9bb0 2024-01-20T08:31:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 >> v1:172.21.15.87:6789/0 conn(0x7f64040c9bb0 legacy=0x7f6404155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec003970 0x7f63ec003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ec0047c0 0x7f63ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f63ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040c4ea0 con 0x7f63ec005380 2024-01-20T08:31:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec00c220 con 0x7f63ec003970 2024-01-20T08:31:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.387+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec00c3e0 con 0x7f63ec0047c0 2024-01-20T08:31:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.390+0000 7f6402d9d640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f63ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.396+0000 7f6402d9d640 1 -- 172.21.15.87:0/765044654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ec0047c0 msgr2=0x7f63ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.396+0000 7f6402d9d640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ec0047c0 0x7f63ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.396+0000 7f6402d9d640 1 -- 172.21.15.87:0/765044654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec003970 msgr2=0x7f63ec003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.396+0000 7f6402d9d640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec003970 0x7f63ec003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.396+0000 7f6402d9d640 1 -- 172.21.15.87:0/765044654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040c4500 con 0x7f63ec005380 2024-01-20T08:31:44.401 INFO:teuthology.orchestra.run.smithi087.stdout:64424509457 2024-01-20T08:31:44.401 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-20T08:31:44.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.397+0000 7f6402d9d640 1 -- 172.21.15.87:0/765044654 reap_dead start 2024-01-20T08:31:44.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.399+0000 7f6402d9d640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f63ec00b980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63fc001d70 tx=0x7f63ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badf92f0ecd4998a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.404+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc034070 con 0x7f63ec005380 2024-01-20T08:31:44.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.404+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc003970 con 0x7f63ec005380 2024-01-20T08:31:44.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.404+0000 7f640159a640 1 -- 172.21.15.87:0/765044654 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc003d20 con 0x7f63ec005380 2024-01-20T08:31:44.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.404+0000 7fa2ef1d7640 1 Processor -- start 2024-01-20T08:31:44.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 -- 172.21.15.87:0/765044654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 msgr2=0x7f63ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f63ec00b980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63fc001d70 tx=0x7f63ec0057f0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7fa2ef1d7640 1 -- start start 2024-01-20T08:31:44.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 -- 172.21.15.87:0/765044654 shutdown_connections 2024-01-20T08:31:44.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 --2- 172.21.15.87:0/765044654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f63ec00b980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 -- 172.21.15.87:0/765044654 >> 172.21.15.87:0/765044654 conn(0x7f640406d410 msgr2=0x7f640406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 -- 172.21.15.87:0/765044654 shutdown_connections 2024-01-20T08:31:44.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.405+0000 7f640359e640 1 -- 172.21.15.87:0/765044654 wait complete. 2024-01-20T08:31:44.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.406+0000 7fa2ef1d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e8150730 0x7fa2e80c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.406+0000 7fa2ef1d7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa2e806e3f0 0x7fa2e80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.406+0000 7fa2ef1d7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e8156d30 con 0x7fa2e81571c0 2024-01-20T08:31:44.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.406+0000 7fa2ef1d7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e8156460 con 0x7fa2e806e3f0 2024-01-20T08:31:44.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.406+0000 7fa2ef1d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e806ecf0 con 0x7fa2e8150730 2024-01-20T08:31:44.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.407+0000 7f640359e640 1 Processor -- start 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.407+0000 7f640359e640 1 -- start start 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7f640359e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f640413f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7f640359e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64041392a0 0x7f6404142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7f640359e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6404142a40 0x7f64040c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7f640359e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040c0240 con 0x7f63ec005380 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7f640359e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040c0520 con 0x7f6404142a40 2024-01-20T08:31:44.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7f640359e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64040c0800 con 0x7f64041392a0 2024-01-20T08:31:44.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.408+0000 7fa2ed774640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa2e806e3f0 0x7fa2e80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.409+0000 7fa2ed774640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa2e806e3f0 0x7fa2e80c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37876/0 (socket says 172.21.15.87:37876) 2024-01-20T08:31:44.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.409+0000 7fa2ed774640 1 -- 172.21.15.87:0/1705863022 learned_addr learned my addr 172.21.15.87:0/1705863022 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.410+0000 7f640259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f640413f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.410+0000 7f640259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f640413f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37890/0 (socket says 172.21.15.87:37890) 2024-01-20T08:31:44.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.410+0000 7f640259c640 1 -- 172.21.15.87:0/1504920479 learned_addr learned my addr 172.21.15.87:0/1504920479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.412+0000 7f164553a640 1 Processor -- start 2024-01-20T08:31:44.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.412+0000 7f164553a640 1 -- start start 2024-01-20T08:31:44.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7fa2ed774640 1 -- 172.21.15.87:0/1705863022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e8150730 msgr2=0x7fa2e80c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7fa2ed774640 1 --2- 172.21.15.87:0/1705863022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e8150730 0x7fa2e80c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7fa2ed774640 1 -- 172.21.15.87:0/1705863022 >> v1:172.21.15.116:6789/0 conn(0x7fa2e81571c0 legacy=0x7fa2e806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7fa2ed774640 1 -- 172.21.15.87:0/1705863022 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e806efd0 con 0x7fa2e806e3f0 2024-01-20T08:31:44.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7f164553a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1640153740 0x7f16400c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7f164553a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16400c2d10 0x7f164006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7f164553a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f164006f0a0 0x7f16400c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7f164553a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164006ddd0 con 0x7f16400c2d10 2024-01-20T08:31:44.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7f164553a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164006e350 con 0x7f164006f0a0 2024-01-20T08:31:44.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.413+0000 7f164553a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164006ffc0 con 0x7f1640153740 2024-01-20T08:31:44.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.416+0000 7fa2ed774640 1 --2- 172.21.15.87:0/1705863022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e806e3f0 0x7fa2e80c88f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009e50 tx=0x7fa2e002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0f290b0a6443a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.420+0000 7fa2df7fe640 1 -- 172.21.15.87:0/1705863022 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e002cb00 con 0x7fa2e806e3f0 2024-01-20T08:31:44.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.420+0000 7fa2df7fe640 1 -- 172.21.15.87:0/1705863022 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e002cc60 con 0x7fa2e806e3f0 2024-01-20T08:31:44.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.420+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/1705863022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e806e3f0 msgr2=0x7fa2e80c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.420+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/1705863022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e806e3f0 0x7fa2e80c88f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009e50 tx=0x7fa2e002bf60 comp rx=0 tx=0).stop 2024-01-20T08:31:44.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.420+0000 7fa2df7fe640 1 -- 172.21.15.87:0/1705863022 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00343f0 con 0x7fa2e806e3f0 2024-01-20T08:31:44.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.421+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/1705863022 shutdown_connections 2024-01-20T08:31:44.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.421+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/1705863022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e806e3f0 0x7fa2e80c88f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.421+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/1705863022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e8150730 0x7fa2e80c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.421+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/1705863022 >> 172.21.15.87:0/1705863022 conn(0x7fa2e806b6c0 msgr2=0x7fa2e806baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.421+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/1705863022 shutdown_connections 2024-01-20T08:31:44.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.421+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/1705863022 wait complete. 2024-01-20T08:31:44.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.422+0000 7fa2ef1d7640 1 Processor -- start 2024-01-20T08:31:44.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.422+0000 7fa2ef1d7640 1 -- start start 2024-01-20T08:31:44.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.423+0000 7fa2ef1d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e806e3f0 0x7fa2e8143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.423+0000 7fa2ef1d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 0x7fa2e81477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.423+0000 7fa2ef1d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e81571c0 0x7fa2e80c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.423+0000 7fa2ef1d7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80c34e0 con 0x7fa2e8150730 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.423+0000 7fa2ef1d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80c3c60 con 0x7fa2e81571c0 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.423+0000 7fa2ef1d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80cbbe0 con 0x7fa2e806e3f0 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.424+0000 7f640259c640 1 -- 172.21.15.87:0/1504920479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64041392a0 msgr2=0x7f6404142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.424+0000 7f640259c640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64041392a0 0x7f6404142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.424+0000 7f640259c640 1 -- 172.21.15.87:0/1504920479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6404142a40 msgr2=0x7f64040c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.424+0000 7f640259c640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6404142a40 0x7f64040c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.424+0000 7f640259c640 1 -- 172.21.15.87:0/1504920479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040c0ae0 con 0x7f63ec005380 2024-01-20T08:31:44.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.427+0000 7f640259c640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f640413f6e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b500 tx=0x7f63f800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.432+0000 7f1637fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16400c2d10 0x7f164006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.432+0000 7fa2dffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 0x7fa2e81477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.433+0000 7f1637fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16400c2d10 0x7f164006eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37904/0 (socket says 172.21.15.87:37904) 2024-01-20T08:31:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.433+0000 7f1637fff640 1 -- 172.21.15.87:0/213415924 learned_addr learned my addr 172.21.15.87:0/213415924 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.433+0000 7fa2dffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 0x7fa2e81477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37918/0 (socket says 172.21.15.87:37918) 2024-01-20T08:31:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.433+0000 7fa2dffff640 1 -- 172.21.15.87:0/2524953556 learned_addr learned my addr 172.21.15.87:0/2524953556 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.434+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8010870 con 0x7f63ec005380 2024-01-20T08:31:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.434+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f80109d0 con 0x7f63ec005380 2024-01-20T08:31:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.434+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8010d80 con 0x7f63ec005380 2024-01-20T08:31:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.434+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64040c0dc0 con 0x7f63ec005380 2024-01-20T08:31:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.434+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6404165d40 con 0x7f63ec005380 2024-01-20T08:31:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.436+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64040c12e0 con 0x7f63ec005380 2024-01-20T08:31:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.440+0000 7fa2dffff640 1 -- 172.21.15.87:0/2524953556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e806e3f0 msgr2=0x7fa2e8143430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.440+0000 7fa2dffff640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e806e3f0 0x7fa2e8143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.440+0000 7fa2dffff640 1 -- 172.21.15.87:0/2524953556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e81571c0 msgr2=0x7fa2e80c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.440+0000 7fa2dffff640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e81571c0 0x7fa2e80c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.440+0000 7fa2dffff640 1 -- 172.21.15.87:0/2524953556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0009b30 con 0x7fa2e8150730 2024-01-20T08:31:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.441+0000 7fc139010640 1 Processor -- start 2024-01-20T08:31:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.441+0000 7fc139010640 1 -- start start 2024-01-20T08:31:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.442+0000 7fc139010640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc134066720 0x7fc1340c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.442+0000 7fc139010640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1341572c0 con 0x7fc1341505d0 2024-01-20T08:31:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.442+0000 7fc139010640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc134156740 con 0x7fc1340c54e0 2024-01-20T08:31:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.442+0000 7fc139010640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340c58f0 con 0x7fc134066720 2024-01-20T08:31:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.443+0000 7f1637fff640 1 -- 172.21.15.87:0/213415924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1640153740 msgr2=0x7f16400c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.444+0000 7f1637fff640 1 --2- 172.21.15.87:0/213415924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1640153740 0x7f16400c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.444+0000 7f1637fff640 1 -- 172.21.15.87:0/213415924 >> v2:172.21.15.116:3300/0 conn(0x7f164006f0a0 msgr2=0x7f16400c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.444+0000 7f1637fff640 1 --2- 172.21.15.87:0/213415924 >> v2:172.21.15.116:3300/0 conn(0x7f164006f0a0 0x7f16400c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.444+0000 7f1637fff640 1 -- 172.21.15.87:0/213415924 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640070280 con 0x7f16400c2d10 2024-01-20T08:31:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.447+0000 7f0f91d25640 1 Processor -- start 2024-01-20T08:31:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.447+0000 7f0f91d25640 1 -- start start 2024-01-20T08:31:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.448+0000 7f0f91d25640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0f8c1506f0 0x7f0f8c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.448+0000 7f0f91d25640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0c0f20 con 0x7f0f8c156430 2024-01-20T08:31:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.448+0000 7f0f91d25640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c156da0 con 0x7f0f8c157920 2024-01-20T08:31:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.448+0000 7f0f91d25640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c91c0 con 0x7f0f8c1506f0 2024-01-20T08:31:44.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.523+0000 7f69e53d1640 1 Processor -- start 2024-01-20T08:31:44.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.524+0000 7f69e53d1640 1 -- start start 2024-01-20T08:31:44.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.524+0000 7f69e53d1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e00c98d0 0x7f69e00cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.524+0000 7f69e53d1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e006bf00 con 0x7f69e00cb930 2024-01-20T08:31:44.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.524+0000 7f69e53d1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e00c9cf0 con 0x7f69e0153af0 2024-01-20T08:31:44.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.524+0000 7f69e53d1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e00c3e10 con 0x7f69e00c98d0 2024-01-20T08:31:44.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.535+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f330407aa60 con 0x7f33140cbd90 2024-01-20T08:31:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.545+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3304076da0 con 0x7f33140cbd90 2024-01-20T08:31:44.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.563+0000 7f69de7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e00c98d0 0x7f69e00cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.563+0000 7fc13259c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc134066720 0x7fc1340c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.563+0000 7f69de7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e00c98d0 0x7f69e00cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50300/0 (socket says 172.21.15.87:50300) 2024-01-20T08:31:44.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.563+0000 7f69de7fc640 1 -- 172.21.15.87:0/2794433641 learned_addr learned my addr 172.21.15.87:0/2794433641 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.564+0000 7fc13259c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc134066720 0x7fc1340c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50298/0 (socket says 172.21.15.87:50298) 2024-01-20T08:31:44.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.564+0000 7fc13259c640 1 -- 172.21.15.87:0/105143859 learned_addr learned my addr 172.21.15.87:0/105143859 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.566+0000 7f69de7fc640 1 -- 172.21.15.87:0/2794433641 >> v1:172.21.15.116:6789/0 conn(0x7f69e0153af0 legacy=0x7f69e006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.566+0000 7f69de7fc640 1 -- 172.21.15.87:0/2794433641 >> v1:172.21.15.87:6789/0 conn(0x7f69e00cb930 legacy=0x7f69e0164090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.566+0000 7f69de7fc640 1 -- 172.21.15.87:0/2794433641 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00cb140 con 0x7f69e00c98d0 2024-01-20T08:31:44.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.569+0000 7fc13259c640 1 -- 172.21.15.87:0/105143859 >> v1:172.21.15.116:6789/0 conn(0x7fc1340c54e0 legacy=0x7fc1340c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.569+0000 7fc13259c640 1 -- 172.21.15.87:0/105143859 >> v1:172.21.15.87:6789/0 conn(0x7fc1341505d0 legacy=0x7fc1340c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:44.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.569+0000 7fc13259c640 1 -- 172.21.15.87:0/105143859 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc134156a20 con 0x7fc134066720 2024-01-20T08:31:44.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.571+0000 7f69de7fc640 1 --2- 172.21.15.87:0/2794433641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00c98d0 0x7f69e00cb3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009850 tx=0x7f69d002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd21927499a9a823 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.575+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f6ad8058170 con 0x7f6ae00e2da0 2024-01-20T08:31:44.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.579+0000 7fc13259c640 1 --2- 172.21.15.87:0/105143859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc134066720 0x7fc1340c4fa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc128009870 tx=0x7fc12802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea18245785970a0d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.585+0000 7f0f8affd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0f8c157920 0x7f0f8c0c1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:39028/0 (socket says 172.21.15.87:39028) 2024-01-20T08:31:44.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.585+0000 7f0f8affd640 1 -- 172.21.15.87:0/965433952 learned_addr learned my addr 172.21.15.87:0/965433952 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:44.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.586+0000 7f69ddffb640 1 -- 172.21.15.87:0/2794433641 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0004230 con 0x7f69e00c98d0 2024-01-20T08:31:44.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.586+0000 7f69ddffb640 1 -- 172.21.15.87:0/2794433641 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d0004390 con 0x7f69e00c98d0 2024-01-20T08:31:44.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.586+0000 7f69ddffb640 1 -- 172.21.15.87:0/2794433641 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0030840 con 0x7f69e00c98d0 2024-01-20T08:31:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 -- 172.21.15.87:0/2794433641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00c98d0 msgr2=0x7f69e00cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2794433641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00c98d0 0x7f69e00cb3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009850 tx=0x7f69d002b980 comp rx=0 tx=0).stop 2024-01-20T08:31:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 -- 172.21.15.87:0/2794433641 shutdown_connections 2024-01-20T08:31:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2794433641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00c98d0 0x7f69e00cb3f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 -- 172.21.15.87:0/2794433641 >> 172.21.15.87:0/2794433641 conn(0x7f69e008b620 msgr2=0x7f69e0071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 -- 172.21.15.87:0/2794433641 shutdown_connections 2024-01-20T08:31:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.587+0000 7f69e53d1640 1 -- 172.21.15.87:0/2794433641 wait complete. 2024-01-20T08:31:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.588+0000 7f69e53d1640 1 Processor -- start 2024-01-20T08:31:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.588+0000 7f69e53d1640 1 -- start start 2024-01-20T08:31:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.588+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6ad8053db0 con 0x7f6ae00e2da0 2024-01-20T08:31:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7fc131d9b640 1 -- 172.21.15.87:0/105143859 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128030860 con 0x7fc134066720 2024-01-20T08:31:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7fc131d9b640 1 -- 172.21.15.87:0/105143859 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280309c0 con 0x7fc134066720 2024-01-20T08:31:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7fc131d9b640 1 -- 172.21.15.87:0/105143859 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128030d50 con 0x7fc134066720 2024-01-20T08:31:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7f69e53d1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00c98d0 0x7f69e00b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7f69e53d1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 0x7f69e0149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7f69e53d1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f69e0153af0 0x7f69e015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7f69e53d1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015be70 con 0x7f69e00c98d0 2024-01-20T08:31:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7f69e53d1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015aa70 con 0x7f69e00cb930 2024-01-20T08:31:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.589+0000 7f69e53d1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015ad50 con 0x7f69e0153af0 2024-01-20T08:31:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 -- 172.21.15.87:0/105143859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc134066720 msgr2=0x7fc1340c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 --2- 172.21.15.87:0/105143859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc134066720 0x7fc1340c4fa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc128009870 tx=0x7fc12802b9a0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 -- 172.21.15.87:0/105143859 shutdown_connections 2024-01-20T08:31:44.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 --2- 172.21.15.87:0/105143859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc134066720 0x7fc1340c4fa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 -- 172.21.15.87:0/105143859 >> 172.21.15.87:0/105143859 conn(0x7fc1340cb3d0 msgr2=0x7fc1340cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 -- 172.21.15.87:0/105143859 shutdown_connections 2024-01-20T08:31:44.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.590+0000 7fc139010640 1 -- 172.21.15.87:0/105143859 wait complete. 2024-01-20T08:31:44.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.591+0000 7f69de7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 0x7f69e0149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.591+0000 7fc139010640 1 Processor -- start 2024-01-20T08:31:44.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.591+0000 7fc139010640 1 -- start start 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7fc139010640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc134066720 0x7fc134137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7fc139010640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1340c54e0 0x7fc1341eda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7fc139010640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 0x7fc1341f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7fc139010640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc134142030 con 0x7fc134066720 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7fc139010640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc134138310 con 0x7fc1341505d0 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7fc139010640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341edf80 con 0x7fc1340c54e0 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7f69de7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 0x7f69e0149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50310/0 (socket says 172.21.15.87:50310) 2024-01-20T08:31:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.592+0000 7f69de7fc640 1 -- 172.21.15.87:0/2371662884 learned_addr learned my addr 172.21.15.87:0/2371662884 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.593+0000 7fc13359e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 0x7fc1341f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.596+0000 7f69de7fc640 1 -- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f69e0153af0 msgr2=0x7f69e015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.596+0000 7f69de7fc640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f69e0153af0 0x7f69e015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.596+0000 7f69de7fc640 1 -- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00c98d0 msgr2=0x7f69e00b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.596+0000 7f69de7fc640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00c98d0 0x7f69e00b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.596+0000 7f69de7fc640 1 -- 172.21.15.87:0/2371662884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0009460 con 0x7f69e00cb930 2024-01-20T08:31:44.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.598+0000 7fc13359e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 0x7fc1341f0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:50322/0 (socket says 172.21.15.87:50322) 2024-01-20T08:31:44.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.598+0000 7fc13359e640 1 -- 172.21.15.87:0/3652695359 learned_addr learned my addr 172.21.15.87:0/3652695359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.599+0000 7f69de7fc640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 0x7f69e0149bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69d00090b0 tx=0x7f69d00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.602+0000 7fc13359e640 1 -- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1340c54e0 msgr2=0x7fc1341eda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.602+0000 7fc13359e640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1340c54e0 0x7fc1341eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.602+0000 7fc13359e640 1 -- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc134066720 msgr2=0x7fc134137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.602+0000 7fc13359e640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc134066720 0x7fc134137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.602+0000 7fc13359e640 1 -- 172.21.15.87:0/3652695359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280094d0 con 0x7fc1341505d0 2024-01-20T08:31:44.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.605+0000 7fc13359e640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 0x7fc1341f0d60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00c780 tx=0x7fc12c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.608+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d003e070 con 0x7f69e00cb930 2024-01-20T08:31:44.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.608+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d0030b80 con 0x7f69e00cb930 2024-01-20T08:31:44.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.608+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0003610 con 0x7f69e00cb930 2024-01-20T08:31:44.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.608+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e015b030 con 0x7f69e00cb930 2024-01-20T08:31:44.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.608+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e014a100 con 0x7f69e00cb930 2024-01-20T08:31:44.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.610+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e0074290 con 0x7f69e00cb930 2024-01-20T08:31:44.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.706+0000 7fa2dffff640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 0x7fa2e81477c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c440 tx=0x7fa2d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.707+0000 7f1637fff640 1 --2- 172.21.15.87:0/213415924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16400c2d10 0x7f164006eb60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f164006c450 tx=0x7f162802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c546f010d35d311d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.728+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f63f8023480 con 0x7f63ec005380 2024-01-20T08:31:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.728+0000 7f63f2ffd640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63e406af50 0x7f63e406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.728+0000 7f6401d9b640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63e406af50 0x7f63e406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.729+0000 7f6401d9b640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63e406af50 0x7f63e406d410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f63f4001eb0 tx=0x7f63f4001660 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.743+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f331406fe70 con 0x7f33140cbd90 2024-01-20T08:31:44.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.745+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f63f8058c30 con 0x7f63ec005380 2024-01-20T08:31:44.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.745+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f63f8058300 con 0x7f63ec005380 2024-01-20T08:31:44.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.766+0000 7f0f8b7fe640 1 --2- 172.21.15.87:0/965433952 >> v2:172.21.15.87:3300/0 conn(0x7f0f8c1506f0 0x7f0f8c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.767+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8005480 con 0x7fa2e8150730 2024-01-20T08:31:44.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.767+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8011680 con 0x7fa2e8150730 2024-01-20T08:31:44.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.767+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e81eb300 con 0x7fa2e8150730 2024-01-20T08:31:44.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.767+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8011a50 con 0x7fa2e8150730 2024-01-20T08:31:44.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.767+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e814a090 con 0x7fa2e8150730 2024-01-20T08:31:44.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.768+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e8074290 con 0x7fa2e8150730 2024-01-20T08:31:44.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.771+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/965433952 >> v1:172.21.15.116:6789/0 conn(0x7f0f8c157920 legacy=0x7f0f8c0c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.771+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/965433952 >> v1:172.21.15.87:6789/0 conn(0x7f0f8c156430 legacy=0x7f0f8c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.771+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/965433952 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c157060 con 0x7f0f8c1506f0 2024-01-20T08:31:44.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.773+0000 7f0f8b7fe640 1 --2- 172.21.15.87:0/965433952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c15a450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0097a0 tx=0x7f0f7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46c1fd6fb8eff68 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.827+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6ae0178f00 con 0x7f6ae00e2da0 2024-01-20T08:31:44.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.832+0000 7f6addffb640 1 -- 172.21.15.87:0/907608340 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ae0178f00 con 0x7f6ae00e2da0 2024-01-20T08:31:44.835 INFO:teuthology.orchestra.run.smithi087.stdout:55834574861 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.834+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6acc06af00 msgr2=0x7f6acc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.834+0000 7f6aee035640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6acc06af00 0x7f6acc06d3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8064100 tx=0x7f6ae80673c0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.834+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 msgr2=0x7f6ae0175450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.834+0000 7f6aee035640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae0175450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c390 tx=0x7f6ad800c850 comp rx=0 tx=0).stop 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 shutdown_connections 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6acc06af00 0x7f6acc06d3c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6ae00e2da0 0x7f6ae0175450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ae00e27e0 0x7f6ae01789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 --2- 172.21.15.87:0/907608340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ae00d9030 0x7f6ae0036200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 >> 172.21.15.87:0/907608340 conn(0x7f6ae004d4b0 msgr2=0x7f6ae00d8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 shutdown_connections 2024-01-20T08:31:44.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.835+0000 7f6aee035640 1 -- 172.21.15.87:0/907608340 wait complete. 2024-01-20T08:31:44.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574861 for osd.7 2024-01-20T08:31:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.869+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f69d0003770 con 0x7f69e00cb930 2024-01-20T08:31:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.869+0000 7f69c77fe640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f69cc06af50 0x7f69cc06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.869+0000 7f69deffd640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f69cc06af50 0x7f69cc06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.870+0000 7f69deffd640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f69cc06af50 0x7f69cc06d410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009650 tx=0x7f69d4006ba0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.874+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f69d007af40 con 0x7f69e00cb930 2024-01-20T08:31:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.875+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c003d80 con 0x7fc1341505d0 2024-01-20T08:31:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.875+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c00ce90 con 0x7fc1341505d0 2024-01-20T08:31:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.875+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1341f05d0 con 0x7fc1341505d0 2024-01-20T08:31:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.875+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0052b0 con 0x7fc1341505d0 2024-01-20T08:31:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.875+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc134142260 con 0x7fc1341505d0 2024-01-20T08:31:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.876+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc134074290 con 0x7fc1341505d0 2024-01-20T08:31:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.887+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f69d0076b80 con 0x7f69e00cb930 2024-01-20T08:31:44.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.976+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f64040c12e0 con 0x7f63ec005380 2024-01-20T08:31:44.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.984+0000 7f163e7fc640 1 -- 172.21.15.87:0/213415924 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628030880 con 0x7f16400c2d10 2024-01-20T08:31:44.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.984+0000 7f163e7fc640 1 -- 172.21.15.87:0/213415924 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16280309e0 con 0x7f16400c2d10 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.984+0000 7f164553a640 1 -- 172.21.15.87:0/213415924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16400c2d10 msgr2=0x7f164006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.984+0000 7f164553a640 1 --2- 172.21.15.87:0/213415924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16400c2d10 0x7f164006eb60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f164006c450 tx=0x7f162802b9c0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 -- 172.21.15.87:0/213415924 shutdown_connections 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 --2- 172.21.15.87:0/213415924 >> v2:172.21.15.116:3300/0 conn(0x7f164006f0a0 0x7f16400c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 --2- 172.21.15.87:0/213415924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16400c2d10 0x7f164006eb60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 --2- 172.21.15.87:0/213415924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1640153740 0x7f16400c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 -- 172.21.15.87:0/213415924 >> 172.21.15.87:0/213415924 conn(0x7f164006b6c0 msgr2=0x7f164006baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 -- 172.21.15.87:0/213415924 shutdown_connections 2024-01-20T08:31:44.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.985+0000 7f164553a640 1 -- 172.21.15.87:0/213415924 wait complete. 2024-01-20T08:31:44.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 Processor -- start 2024-01-20T08:31:44.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 -- start start 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 0x7f16401464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16400c2d10 0x7f16401e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1640153740 0x7f16401f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16401ef630 con 0x7f1640153740 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164014ac00 con 0x7f16400c2d10 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.986+0000 7f164553a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164014b300 con 0x7f164006f0a0 2024-01-20T08:31:44.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.987+0000 7f163effd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 0x7f16401464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.987+0000 7f163effd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 0x7f16401464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:36546/0 (socket says 172.21.15.87:36546) 2024-01-20T08:31:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.987+0000 7f163effd640 1 -- 172.21.15.87:0/986806332 learned_addr learned my addr 172.21.15.87:0/986806332 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.991+0000 7f163effd640 1 -- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16400c2d10 msgr2=0x7f16401e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.991+0000 7f163effd640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16400c2d10 0x7f16401e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.991+0000 7f163effd640 1 -- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1640153740 msgr2=0x7f16401f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.991+0000 7f163effd640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1640153740 0x7f16401f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.991+0000 7f163effd640 1 -- 172.21.15.87:0/986806332 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280094d0 con 0x7f164006f0a0 2024-01-20T08:31:44.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.994+0000 7f163effd640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 0x7f16401464b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f163000b4d0 tx=0x7f163000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.998+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630010870 con 0x7f164006f0a0 2024-01-20T08:31:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.998+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16300109d0 con 0x7f164006f0a0 2024-01-20T08:31:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.998+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16401efb40 con 0x7f164006f0a0 2024-01-20T08:31:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.998+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630010d80 con 0x7f164006f0a0 2024-01-20T08:31:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.998+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164014a890 con 0x7f164006f0a0 2024-01-20T08:31:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:44.999+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164006e7c0 con 0x7f164006f0a0 2024-01-20T08:31:45.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.016+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1630023480 con 0x7f164006f0a0 2024-01-20T08:31:45.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.016+0000 7f16377fe640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f162006af20 0x7f162006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.017+0000 7f1637fff640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f162006af20 0x7f162006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.017+0000 7f1637fff640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f162006af20 0x7f162006d3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1628036000 tx=0x7f1628030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.019+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f1630058b00 con 0x7f164006f0a0 2024-01-20T08:31:45.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.047+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa2d8017070 con 0x7fa2e8150730 2024-01-20T08:31:45.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.047+0000 7fa2dd7fa640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2d406af00 0x7fa2d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.047+0000 7fa2ecf73640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2d406af00 0x7fa2d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.048+0000 7fa2ecf73640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2d406af00 0x7fa2d406d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009670 tx=0x7fa2e4006bf0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.062+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fa2d8021300 con 0x7fa2e8150730 2024-01-20T08:31:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.062+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa2d805bd40 con 0x7fa2e8150730 2024-01-20T08:31:45.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.069+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f69e015abb0 con 0x7f69e00cb930 2024-01-20T08:31:45.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.117+0000 7f0f8a7fc640 1 -- 172.21.15.87:0/965433952 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c031830 con 0x7f0f8c1506f0 2024-01-20T08:31:45.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.117+0000 7f0f8a7fc640 1 -- 172.21.15.87:0/965433952 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c031990 con 0x7f0f8c1506f0 2024-01-20T08:31:45.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.117+0000 7f0f8a7fc640 1 -- 172.21.15.87:0/965433952 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c031d20 con 0x7f0f8c1506f0 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.117+0000 7f0f91d25640 1 -- 172.21.15.87:0/965433952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 msgr2=0x7f0f8c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.117+0000 7f0f91d25640 1 --2- 172.21.15.87:0/965433952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c15a450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0097a0 tx=0x7f0f7c02b8d0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.118+0000 7f0f91d25640 1 -- 172.21.15.87:0/965433952 shutdown_connections 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.118+0000 7f0f91d25640 1 --2- 172.21.15.87:0/965433952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c15a450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.118+0000 7f0f91d25640 1 -- 172.21.15.87:0/965433952 >> 172.21.15.87:0/965433952 conn(0x7f0f8c072810 msgr2=0x7f0f8c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.118+0000 7f0f91d25640 1 -- 172.21.15.87:0/965433952 shutdown_connections 2024-01-20T08:31:45.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.118+0000 7f0f91d25640 1 -- 172.21.15.87:0/965433952 wait complete. 2024-01-20T08:31:45.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.119+0000 7f0f91d25640 1 Processor -- start 2024-01-20T08:31:45.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.119+0000 7f0f91d25640 1 -- start start 2024-01-20T08:31:45.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.120+0000 7f0f91d25640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c1edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.120+0000 7f0f91d25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f8c156430 0x7f0f8c144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.120+0000 7f0f91d25640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0f8c157920 0x7f0f8c158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.120+0000 7f0f91d25640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c68a0 con 0x7f0f8c1506f0 2024-01-20T08:31:45.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.120+0000 7f0f91d25640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1ee210 con 0x7f0f8c156430 2024-01-20T08:31:45.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.120+0000 7f0f91d25640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1410b0 con 0x7f0f8c157920 2024-01-20T08:31:45.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.130+0000 7f63f2ffd640 1 -- 172.21.15.87:0/1504920479 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64040c12e0 con 0x7f63ec005380 2024-01-20T08:31:45.134 INFO:teuthology.orchestra.run.smithi087.stdout:60129542158 2024-01-20T08:31:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.132+0000 7f0f8b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c1edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.132+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63e406af50 msgr2=0x7f63e406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.132+0000 7f640359e640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63e406af50 0x7f63e406d410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f63f4001eb0 tx=0x7f63f4001660 comp rx=0 tx=0).stop 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 msgr2=0x7f640413f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f640413f6e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b500 tx=0x7f63f800b9c0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 shutdown_connections 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63e406af50 0x7f63e406d410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6404142a40 0x7f64040c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64041392a0 0x7f6404142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 --2- 172.21.15.87:0/1504920479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63ec005380 0x7f640413f6e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 >> 172.21.15.87:0/1504920479 conn(0x7f640406d410 msgr2=0x7f6404154f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 shutdown_connections 2024-01-20T08:31:45.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.133+0000 7f640359e640 1 -- 172.21.15.87:0/1504920479 wait complete. 2024-01-20T08:31:45.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.134+0000 7f0f8b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c1edcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37958/0 (socket says 172.21.15.87:37958) 2024-01-20T08:31:45.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.134+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/1724281248 learned_addr learned my addr 172.21.15.87:0/1724281248 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.139+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/1724281248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0f8c157920 msgr2=0x7f0f8c158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.139+0000 7f0f8b7fe640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0f8c157920 0x7f0f8c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.139+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/1724281248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f8c156430 msgr2=0x7f0f8c144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.139+0000 7f0f8b7fe640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f8c156430 0x7f0f8c144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.139+0000 7f0f8b7fe640 1 -- 172.21.15.87:0/1724281248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009400 con 0x7f0f8c1506f0 2024-01-20T08:31:45.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.142+0000 7f0f8b7fe640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c1edcd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c034040 tx=0x7f0f7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542159 got 60129542158 for osd.5 2024-01-20T08:31:45.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.147+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c03aa60 con 0x7f0f8c1506f0 2024-01-20T08:31:45.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.147+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c03abe0 con 0x7f0f8c1506f0 2024-01-20T08:31:45.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.148+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0cc5f0 con 0x7f0f8c1506f0 2024-01-20T08:31:45.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.148+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c044400 con 0x7f0f8c1506f0 2024-01-20T08:31:45.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.148+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c0c6300 con 0x7f0f8c1506f0 2024-01-20T08:31:45.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.148+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c0cc450 con 0x7f0f8c1506f0 2024-01-20T08:31:45.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.161+0000 7f330a7fc640 1 -- 172.21.15.87:0/915100945 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3304049090 con 0x7f33140cbd90 2024-01-20T08:31:45.164 INFO:teuthology.orchestra.run.smithi087.stdout:60129542158 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f32f806aed0 msgr2=0x7f32f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f32f806aed0 0x7f32f806d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3310005880 tx=0x7f3310001150 comp rx=0 tx=0).stop 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 msgr2=0x7f33140c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 0x7f33140c44f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f330402bf00 tx=0x7f33040046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 shutdown_connections 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f32f806aed0 0x7f32f806d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33140cc170 0x7f33140c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33140cbd90 0x7f33140c44f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 --2- 172.21.15.87:0/915100945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33140703c0 0x7f331414b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 >> 172.21.15.87:0/915100945 conn(0x7f3314071e50 msgr2=0x7f33140ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 shutdown_connections 2024-01-20T08:31:45.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.163+0000 7f331a9bc640 1 -- 172.21.15.87:0/915100945 wait complete. 2024-01-20T08:31:45.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542159 got 60129542158 for osd.2 2024-01-20T08:31:45.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.177+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc12c005410 con 0x7fc1341505d0 2024-01-20T08:31:45.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.177+0000 7fc11b7fe640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc12006af20 0x7fc12006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.178+0000 7fc132d9d640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc12006af20 0x7fc12006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.178+0000 7fc132d9d640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc12006af20 0x7fc12006d3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc124009650 tx=0x7fc124006ba0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.180+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fc12c058660 con 0x7fc1341505d0 2024-01-20T08:31:45.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.194+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc12c0542a0 con 0x7fc1341505d0 2024-01-20T08:31:45.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.248+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa2e8155ab0 con 0x7fa2e8150730 2024-01-20T08:31:45.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.280+0000 7f69c77fe640 1 -- 172.21.15.87:0/2371662884 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69e015abb0 con 0x7f69e00cb930 2024-01-20T08:31:45.283 INFO:teuthology.orchestra.run.smithi087.stdout:64424509455 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f69cc06af50 msgr2=0x7f69cc06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f69cc06af50 0x7f69cc06d410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009650 tx=0x7f69d4006ba0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 msgr2=0x7f69e0149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 0x7f69e0149bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69d00090b0 tx=0x7f69d00046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 shutdown_connections 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f69cc06af50 0x7f69cc06d410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f69e0153af0 0x7f69e015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e00cb930 0x7f69e0149bc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 --2- 172.21.15.87:0/2371662884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00c98d0 0x7f69e00b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 >> 172.21.15.87:0/2371662884 conn(0x7f69e008b620 msgr2=0x7f69e014e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 shutdown_connections 2024-01-20T08:31:45.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.282+0000 7f69e53d1640 1 -- 172.21.15.87:0/2371662884 wait complete. 2024-01-20T08:31:45.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509457 got 64424509455 for osd.0 2024-01-20T08:31:45.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.312+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1630054740 con 0x7f164006f0a0 2024-01-20T08:31:45.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.372+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc1340c2560 con 0x7fc1341505d0 2024-01-20T08:31:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.375+0000 7fc11b7fe640 1 -- 172.21.15.87:0/3652695359 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc12c0578c0 con 0x7fc1341505d0 2024-01-20T08:31:45.378 INFO:teuthology.orchestra.run.smithi087.stdout:55834574861 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.377+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc12006af20 msgr2=0x7fc12006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc12006af20 0x7fc12006d3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc124009650 tx=0x7fc124006ba0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 msgr2=0x7fc1341f0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 0x7fc1341f0d60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00c780 tx=0x7fc12c00cc40 comp rx=0 tx=0).stop 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 shutdown_connections 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc12006af20 0x7fc12006d3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1341505d0 0x7fc1341f0d60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1340c54e0 0x7fc1341eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 --2- 172.21.15.87:0/3652695359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc134066720 0x7fc134137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 >> 172.21.15.87:0/3652695359 conn(0x7fc1340cb3d0 msgr2=0x7fc134144bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 shutdown_connections 2024-01-20T08:31:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.378+0000 7fc139010640 1 -- 172.21.15.87:0/3652695359 wait complete. 2024-01-20T08:31:45.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574861 for osd.3 2024-01-20T08:31:45.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.417+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0f7c04d440 con 0x7f0f8c1506f0 2024-01-20T08:31:45.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.417+0000 7f0f73fff640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0f7806af50 0x7f0f7806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.417+0000 7f0f8affd640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0f7806af50 0x7f0f7806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.418+0000 7f0f8affd640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0f7806af50 0x7f0f7806d410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c158a30 tx=0x7f0f80007a40 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.445+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0f7c07ae80 con 0x7f0f8c1506f0 2024-01-20T08:31:45.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.445+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0f7c076ac0 con 0x7f0f8c1506f0 2024-01-20T08:31:45.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.484+0000 7fa2dd7fa640 1 -- 172.21.15.87:0/2524953556 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa2d805b8d0 con 0x7fa2e8150730 2024-01-20T08:31:45.488 INFO:teuthology.orchestra.run.smithi087.stdout:64424509455 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2d406af00 msgr2=0x7fa2d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2d406af00 0x7fa2d406d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009670 tx=0x7fa2e4006bf0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 msgr2=0x7fa2e81477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 0x7fa2e81477c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c440 tx=0x7fa2d800c900 comp rx=0 tx=0).stop 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 shutdown_connections 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2d406af00 0x7fa2d406d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e81571c0 0x7fa2e80c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e8150730 0x7fa2e81477c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 --2- 172.21.15.87:0/2524953556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2e806e3f0 0x7fa2e8143430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 >> 172.21.15.87:0/2524953556 conn(0x7fa2e806b6c0 msgr2=0x7fa2e806f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 shutdown_connections 2024-01-20T08:31:45.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.487+0000 7fa2ef1d7640 1 -- 172.21.15.87:0/2524953556 wait complete. 2024-01-20T08:31:45.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509457 got 64424509455 for osd.6 2024-01-20T08:31:45.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.563+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f164006e7c0 con 0x7f164006f0a0 2024-01-20T08:31:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.566+0000 7f16377fe640 1 -- 172.21.15.87:0/986806332 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f163001e020 con 0x7f164006f0a0 2024-01-20T08:31:45.569 INFO:teuthology.orchestra.run.smithi087.stdout:60129542158 2024-01-20T08:31:45.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.569+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f162006af20 msgr2=0x7f162006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.569+0000 7f164553a640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f162006af20 0x7f162006d3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1628036000 tx=0x7f1628030000 comp rx=0 tx=0).stop 2024-01-20T08:31:45.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.569+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 msgr2=0x7f16401464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.569+0000 7f164553a640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 0x7f16401464b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f163000b4d0 tx=0x7f163000b990 comp rx=0 tx=0).stop 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 shutdown_connections 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f162006af20 0x7f162006d3e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1640153740 0x7f16401f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16400c2d10 0x7f16401e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 --2- 172.21.15.87:0/986806332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f164006f0a0 0x7f16401464b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 >> 172.21.15.87:0/986806332 conn(0x7f164006b6c0 msgr2=0x7f1640150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 shutdown_connections 2024-01-20T08:31:45.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.570+0000 7f164553a640 1 -- 172.21.15.87:0/986806332 wait complete. 2024-01-20T08:31:45.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542158 got 60129542158 for osd.4 2024-01-20T08:31:45.587 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:45.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.662+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0f8c0cc450 con 0x7f0f8c1506f0 2024-01-20T08:31:45.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.666+0000 7f0f73fff640 1 -- 172.21.15.87:0/1724281248 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f8c0cc450 con 0x7f0f8c1506f0 2024-01-20T08:31:45.669 INFO:teuthology.orchestra.run.smithi087.stdout:60129542159 2024-01-20T08:31:45.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0f7806af50 msgr2=0x7f0f7806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0f7806af50 0x7f0f7806d410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c158a30 tx=0x7f0f80007a40 comp rx=0 tx=0).stop 2024-01-20T08:31:45.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 msgr2=0x7f0f8c1edcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c1edcd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c034040 tx=0x7f0f7c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 shutdown_connections 2024-01-20T08:31:45.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0f7806af50 0x7f0f7806d410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0f8c157920 0x7f0f8c158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f8c156430 0x7f0f8c144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 --2- 172.21.15.87:0/1724281248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f8c1506f0 0x7f0f8c1edcd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 >> 172.21.15.87:0/1724281248 conn(0x7f0f8c072810 msgr2=0x7f0f8c154270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 shutdown_connections 2024-01-20T08:31:45.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.669+0000 7f0f91d25640 1 -- 172.21.15.87:0/1724281248 wait complete. 2024-01-20T08:31:45.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542160 got 60129542159 for osd.1 2024-01-20T08:31:45.853 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-20T08:31:45.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.951+0000 7f67912da640 1 Processor -- start 2024-01-20T08:31:45.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.951+0000 7f67912da640 1 -- start start 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.952+0000 7f67912da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151640 0x7f678c06aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.952+0000 7f67912da640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f678c151a20 0x7f678c06af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.952+0000 7f67912da640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c06a5b0 con 0x7f678c0c5a90 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.952+0000 7f67912da640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c15b5e0 con 0x7f678c151a20 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.952+0000 7f67912da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0c6760 con 0x7f678c151640 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.953+0000 7f678b7fe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f678c0c5a90 0x7f678c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33546/0 (socket says 172.21.15.87:33546) 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.953+0000 7f678b7fe640 1 -- 172.21.15.87:0/43155943 learned_addr learned my addr 172.21.15.87:0/43155943 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.953+0000 7f678affd640 1 --2- 172.21.15.87:0/43155943 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151640 0x7f678c06aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.953+0000 7f678a7fc640 1 --2- 172.21.15.87:0/43155943 >> v2:172.21.15.116:3300/0 conn(0x7f678c151a20 0x7f678c06af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.955+0000 7f6789ffb640 1 -- 172.21.15.87:0/43155943 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199997839 0 0) 0x7f678c06a5b0 con 0x7f678c0c5a90 2024-01-20T08:31:45.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.955+0000 7f6789ffb640 1 -- 172.21.15.87:0/43155943 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6774003520 con 0x7f678c0c5a90 2024-01-20T08:31:45.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.955+0000 7f678affd640 1 -- 172.21.15.87:0/43155943 >> v2:172.21.15.116:3300/0 conn(0x7f678c151a20 msgr2=0x7f678c06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:45.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.955+0000 7f678affd640 1 --2- 172.21.15.87:0/43155943 >> v2:172.21.15.116:3300/0 conn(0x7f678c151a20 0x7f678c06af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.955+0000 7f678affd640 1 -- 172.21.15.87:0/43155943 >> v1:172.21.15.87:6789/0 conn(0x7f678c0c5a90 legacy=0x7f678c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.955+0000 7f678affd640 1 -- 172.21.15.87:0/43155943 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c15b880 con 0x7f678c151640 2024-01-20T08:31:45.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.957+0000 7f678affd640 1 --2- 172.21.15.87:0/43155943 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151640 0x7f678c06aa40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f677c009d90 tx=0x7f677c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1857f0348956a74c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.961+0000 7f6789ffb640 1 -- 172.21.15.87:0/43155943 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c02cce0 con 0x7f678c151640 2024-01-20T08:31:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.961+0000 7f6789ffb640 1 -- 172.21.15.87:0/43155943 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c02ce40 con 0x7f678c151640 2024-01-20T08:31:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.961+0000 7f6789ffb640 1 -- 172.21.15.87:0/43155943 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c034660 con 0x7f678c151640 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 -- 172.21.15.87:0/43155943 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151640 msgr2=0x7f678c06aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 --2- 172.21.15.87:0/43155943 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151640 0x7f678c06aa40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f677c009d90 tx=0x7f677c02bf30 comp rx=0 tx=0).stop 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 -- 172.21.15.87:0/43155943 shutdown_connections 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 --2- 172.21.15.87:0/43155943 >> v2:172.21.15.116:3300/0 conn(0x7f678c151a20 0x7f678c06af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 --2- 172.21.15.87:0/43155943 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151640 0x7f678c06aa40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 -- 172.21.15.87:0/43155943 >> 172.21.15.87:0/43155943 conn(0x7f678c0723c0 msgr2=0x7f678c06fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 -- 172.21.15.87:0/43155943 shutdown_connections 2024-01-20T08:31:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.962+0000 7f67912da640 1 -- 172.21.15.87:0/43155943 wait complete. 2024-01-20T08:31:45.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.963+0000 7f67912da640 1 Processor -- start 2024-01-20T08:31:45.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.963+0000 7f67912da640 1 -- start start 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f67912da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678c0c5a90 0x7f678c0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f67912da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 0x7f678c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f67912da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151a20 0x7f678c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f67912da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c154d00 con 0x7f678c151640 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f67912da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0c0e80 con 0x7f678c0c5a90 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f67912da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c154960 con 0x7f678c151a20 2024-01-20T08:31:45.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f678a7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 0x7f678c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151a20 0x7f678c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.964+0000 7f678affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678c0c5a90 0x7f678c0c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.965+0000 7f678a7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 0x7f678c152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45590/0 (socket says 172.21.15.87:45590) 2024-01-20T08:31:45.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.965+0000 7f678a7fc640 1 -- 172.21.15.87:0/1415389879 learned_addr learned my addr 172.21.15.87:0/1415389879 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:45.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.968+0000 7f678a7fc640 1 -- 172.21.15.87:0/1415389879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151a20 msgr2=0x7f678c1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.968+0000 7f678a7fc640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151a20 0x7f678c1528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.968+0000 7f678a7fc640 1 -- 172.21.15.87:0/1415389879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678c0c5a90 msgr2=0x7f678c0c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:45.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.968+0000 7f678a7fc640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678c0c5a90 0x7f678c0c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:45.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.968+0000 7f678a7fc640 1 -- 172.21.15.87:0/1415389879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c009a70 con 0x7f678c151640 2024-01-20T08:31:45.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.970+0000 7f678a7fc640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 0x7f678c152370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f678000e840 tx=0x7f678000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:45.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.974+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800094f0 con 0x7f678c151640 2024-01-20T08:31:45.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.974+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000c940 con 0x7f678c151640 2024-01-20T08:31:45.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.974+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000ccd0 con 0x7f678c151640 2024-01-20T08:31:45.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.974+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c154170 con 0x7f678c151640 2024-01-20T08:31:45.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.974+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0c2b30 con 0x7f678c151640 2024-01-20T08:31:45.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.975+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c067890 con 0x7f678c151640 2024-01-20T08:31:45.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.994+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6780013470 con 0x7f678c151640 2024-01-20T08:31:45.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.994+0000 7f67737fe640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f677806aed0 0x7f677806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:45.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.995+0000 7f678affd640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f677806aed0 0x7f677806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:45.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:45.995+0000 7f678affd640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f677806aed0 0x7f677806d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f677c036000 tx=0x7f677c030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.149 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-20T08:31:46.177 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-20T08:31:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.245+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f678000ce30 con 0x7f678c151640 2024-01-20T08:31:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.245+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6780054dd0 con 0x7f678c151640 2024-01-20T08:31:46.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.258+0000 7f905ffb6640 1 Processor -- start 2024-01-20T08:31:46.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.259+0000 7f905ffb6640 1 -- start start 2024-01-20T08:31:46.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.259+0000 7f905ffb6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9058153a20 0x7f905815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.259+0000 7f905ffb6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f90580cefd0 0x7f90580cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.259+0000 7f905ffb6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905815a7f0 con 0x7f90580c9e00 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.259+0000 7f905ffb6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580cc3c0 con 0x7f90580cefd0 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.259+0000 7f905ffb6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580ca780 con 0x7f9058153a20 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.260+0000 7f905d551640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f90580c9e00 0x7f90580c3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:36400/0 (socket says 172.21.15.87:36400) 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.260+0000 7f905d551640 1 -- 172.21.15.87:0/622441962 learned_addr learned my addr 172.21.15.87:0/622441962 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:46.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.260+0000 7f905dd52640 1 --2- 172.21.15.87:0/622441962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9058153a20 0x7f905815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.263+0000 7f905cd50640 1 -- 172.21.15.87:0/622441962 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818699084 0 0) 0x7f9048001110 con 0x7f90580c9e00 2024-01-20T08:31:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.263+0000 7f905cd50640 1 -- 172.21.15.87:0/622441962 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9044003520 con 0x7f90580c9e00 2024-01-20T08:31:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.263+0000 7f905dd52640 1 -- 172.21.15.87:0/622441962 >> v2:172.21.15.87:3300/0 conn(0x7f90580cefd0 msgr2=0x7f90580cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.263+0000 7f905dd52640 1 --2- 172.21.15.87:0/622441962 >> v2:172.21.15.87:3300/0 conn(0x7f90580cefd0 0x7f90580cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.263+0000 7f905dd52640 1 -- 172.21.15.87:0/622441962 >> v1:172.21.15.116:6789/0 conn(0x7f90580c9e00 legacy=0x7f90580c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.263+0000 7f905dd52640 1 -- 172.21.15.87:0/622441962 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580c4670 con 0x7f9058153a20 2024-01-20T08:31:46.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.265+0000 7f905dd52640 1 --2- 172.21.15.87:0/622441962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9058153a20 0x7f905815a2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f90540097a0 tx=0x7f905402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90f40feacaec643 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.268+0000 7f905cd50640 1 -- 172.21.15.87:0/622441962 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905402e070 con 0x7f9058153a20 2024-01-20T08:31:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.268+0000 7f905cd50640 1 -- 172.21.15.87:0/622441962 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90540315a0 con 0x7f9058153a20 2024-01-20T08:31:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.268+0000 7f905cd50640 1 -- 172.21.15.87:0/622441962 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054031930 con 0x7f9058153a20 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 -- 172.21.15.87:0/622441962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9058153a20 msgr2=0x7f905815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 --2- 172.21.15.87:0/622441962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9058153a20 0x7f905815a2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f90540097a0 tx=0x7f905402b840 comp rx=0 tx=0).stop 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 -- 172.21.15.87:0/622441962 shutdown_connections 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 --2- 172.21.15.87:0/622441962 >> v2:172.21.15.87:3300/0 conn(0x7f90580cefd0 0x7f90580cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 --2- 172.21.15.87:0/622441962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9058153a20 0x7f905815a2b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 -- 172.21.15.87:0/622441962 >> 172.21.15.87:0/622441962 conn(0x7f905808af90 msgr2=0x7f905808b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 -- 172.21.15.87:0/622441962 shutdown_connections 2024-01-20T08:31:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.269+0000 7f905ffb6640 1 -- 172.21.15.87:0/622441962 wait complete. 2024-01-20T08:31:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.270+0000 7f905ffb6640 1 Processor -- start 2024-01-20T08:31:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.270+0000 7f905ffb6640 1 -- start start 2024-01-20T08:31:46.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905ffb6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 0x7f9058156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905ffb6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90580cefd0 0x7f90581ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905ffb6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9058153a20 0x7f90581efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905ffb6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c69c0 con 0x7f9058153a20 2024-01-20T08:31:46.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905ffb6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c66e0 con 0x7f90580cefd0 2024-01-20T08:31:46.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905ffb6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90581542d0 con 0x7f90580c9e00 2024-01-20T08:31:46.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905dd52640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 0x7f9058156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.271+0000 7f905d551640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90580cefd0 0x7f90581ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.272+0000 7f905dd52640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 0x7f9058156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47310/0 (socket says 172.21.15.87:47310) 2024-01-20T08:31:46.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.272+0000 7f905dd52640 1 -- 172.21.15.87:0/4037944065 learned_addr learned my addr 172.21.15.87:0/4037944065 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.275+0000 7f905dd52640 1 -- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90580cefd0 msgr2=0x7f90581ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.275+0000 7f905dd52640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90580cefd0 0x7f90581ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.275+0000 7f905dd52640 1 -- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9058153a20 msgr2=0x7f90581efb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.275+0000 7f905dd52640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9058153a20 0x7f90581efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.275+0000 7f905dd52640 1 -- 172.21.15.87:0/4037944065 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9054009400 con 0x7f90580c9e00 2024-01-20T08:31:46.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.277+0000 7f905dd52640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 0x7f9058156980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9054034040 tx=0x7f90540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.281+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054031f10 con 0x7f90580c9e00 2024-01-20T08:31:46.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.281+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9054032640 con 0x7f90580c9e00 2024-01-20T08:31:46.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.281+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90581545b0 con 0x7f90580c9e00 2024-01-20T08:31:46.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.282+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540329f0 con 0x7f90580c9e00 2024-01-20T08:31:46.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.282+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058154b70 con 0x7f90580c9e00 2024-01-20T08:31:46.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.283+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9058074290 con 0x7f90580c9e00 2024-01-20T08:31:46.301 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-20T08:31:46.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.302+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9054032b50 con 0x7f90580c9e00 2024-01-20T08:31:46.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.302+0000 7f904e7fc640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f903c06af20 0x7f903c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.302+0000 7f905d551640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f903c06af20 0x7f903c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.303+0000 7f905d551640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f903c06af20 0x7f903c06d3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9058154050 tx=0x7f9048009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.304+0000 7f0c267b6640 1 Processor -- start 2024-01-20T08:31:46.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.304+0000 7f0c267b6640 1 -- start start 2024-01-20T08:31:46.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.305+0000 7f0c267b6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c200c9300 0x7f0c2006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.305+0000 7f0c267b6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200cc8b0 con 0x7f0c20153b50 2024-01-20T08:31:46.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.305+0000 7f0c267b6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2015cd10 con 0x7f0c200700e0 2024-01-20T08:31:46.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.305+0000 7f0c267b6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200704f0 con 0x7f0c200c9300 2024-01-20T08:31:46.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.306+0000 7f0c24d53640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0c200700e0 0x7f0c20159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:36408/0 (socket says 172.21.15.87:36408) 2024-01-20T08:31:46.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.306+0000 7f0c24d53640 1 -- 172.21.15.87:0/3071287415 learned_addr learned my addr 172.21.15.87:0/3071287415 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:46.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.317+0000 7f0c1f7fe640 1 --2- 172.21.15.87:0/3071287415 >> v2:172.21.15.87:3300/0 conn(0x7f0c200c9300 0x7f0c2006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.393 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-20T08:31:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.393+0000 7f0c1f7fe640 1 -- 172.21.15.87:0/3071287415 >> v1:172.21.15.116:6789/0 conn(0x7f0c200700e0 legacy=0x7f0c20159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.394+0000 7f0c1f7fe640 1 -- 172.21.15.87:0/3071287415 >> v1:172.21.15.87:6789/0 conn(0x7f0c20153b50 legacy=0x7f0c200c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.394+0000 7f0c1f7fe640 1 -- 172.21.15.87:0/3071287415 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2015cfd0 con 0x7f0c200c9300 2024-01-20T08:31:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.403+0000 7f0c1f7fe640 1 --2- 172.21.15.87:0/3071287415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c200c9300 0x7f0c2006fba0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c140097a0 tx=0x7f0c1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a1618f73956c8e server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.406+0000 7f047b50e640 1 Processor -- start 2024-01-20T08:31:46.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.406+0000 7f047b50e640 1 -- start start 2024-01-20T08:31:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.407+0000 7f047b50e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0474151740 0x7f04740c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.407+0000 7f047b50e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0474157370 0x7f047406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.407+0000 7f047b50e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04740c76a0 con 0x7f04740c5d30 2024-01-20T08:31:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.407+0000 7f047b50e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740c70e0 con 0x7f0474157370 2024-01-20T08:31:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.407+0000 7f047b50e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0474151b20 con 0x7f0474151740 2024-01-20T08:31:46.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.409+0000 7f0c1effd640 1 -- 172.21.15.87:0/3071287415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14031830 con 0x7f0c200c9300 2024-01-20T08:31:46.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.409+0000 7f0c1effd640 1 -- 172.21.15.87:0/3071287415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c14031990 con 0x7f0c200c9300 2024-01-20T08:31:46.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.409+0000 7f0c1effd640 1 -- 172.21.15.87:0/3071287415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14031d20 con 0x7f0c200c9300 2024-01-20T08:31:46.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0478aa9640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f04740c5d30 0x7f047406fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33562/0 (socket says 172.21.15.87:33562) 2024-01-20T08:31:46.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0478aa9640 1 -- 172.21.15.87:0/3865126311 learned_addr learned my addr 172.21.15.87:0/3865126311 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:46.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 -- 172.21.15.87:0/3071287415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c200c9300 msgr2=0x7f0c2006fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3071287415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c200c9300 0x7f0c2006fba0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c140097a0 tx=0x7f0c1402b8d0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 -- 172.21.15.87:0/3071287415 shutdown_connections 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3071287415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c200c9300 0x7f0c2006fba0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 -- 172.21.15.87:0/3071287415 >> 172.21.15.87:0/3071287415 conn(0x7f0c20072810 msgr2=0x7f0c2006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 -- 172.21.15.87:0/3071287415 shutdown_connections 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.410+0000 7f0c267b6640 1 -- 172.21.15.87:0/3071287415 wait complete. 2024-01-20T08:31:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.411+0000 7f0479aab640 1 --2- 172.21.15.87:0/3865126311 >> v2:172.21.15.87:3300/0 conn(0x7f0474157370 0x7f047406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.412+0000 7f0c267b6640 1 Processor -- start 2024-01-20T08:31:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.412+0000 7f0c267b6640 1 -- start start 2024-01-20T08:31:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.413+0000 7f0c267b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c200700e0 0x7f0c201e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.413+0000 7f0c267b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c200c9300 0x7f0c201ed790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.413+0000 7f0c267b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 0x7f0c201ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.413+0000 7f0c267b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201e60c0 con 0x7f0c20153b50 2024-01-20T08:31:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.413+0000 7f0c267b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201f1910 con 0x7f0c200700e0 2024-01-20T08:31:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.413+0000 7f0c267b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201f1bf0 con 0x7f0c200c9300 2024-01-20T08:31:46.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.416+0000 7f0479aab640 1 -- 172.21.15.87:0/3865126311 >> v2:172.21.15.116:3300/0 conn(0x7f0474151740 msgr2=0x7f04740c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.416+0000 7f0479aab640 1 --2- 172.21.15.87:0/3865126311 >> v2:172.21.15.116:3300/0 conn(0x7f0474151740 0x7f04740c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.416+0000 7f0479aab640 1 -- 172.21.15.87:0/3865126311 >> v1:172.21.15.87:6789/0 conn(0x7f04740c5d30 legacy=0x7f047406fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.416+0000 7f0479aab640 1 -- 172.21.15.87:0/3865126311 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04740c88f0 con 0x7f0474157370 2024-01-20T08:31:46.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.419+0000 7f0479aab640 1 --2- 172.21.15.87:0/3865126311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f047406e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04740c31a0 tx=0x7f046c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31f61c1761b9f12 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.423+0000 7f0c24d53640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 0x7f0c201ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.424+0000 7f0c24d53640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 0x7f0c201ef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45630/0 (socket says 172.21.15.87:45630) 2024-01-20T08:31:46.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.424+0000 7f0c24d53640 1 -- 172.21.15.87:0/3596907494 learned_addr learned my addr 172.21.15.87:0/3596907494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.425+0000 7f046bfff640 1 -- 172.21.15.87:0/3865126311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c02cc80 con 0x7f0474157370 2024-01-20T08:31:46.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.426+0000 7f046bfff640 1 -- 172.21.15.87:0/3865126311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c02cde0 con 0x7f0474157370 2024-01-20T08:31:46.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.426+0000 7f046bfff640 1 -- 172.21.15.87:0/3865126311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c034660 con 0x7f0474157370 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.426+0000 7f047b50e640 1 -- 172.21.15.87:0/3865126311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 msgr2=0x7f047406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.426+0000 7f047b50e640 1 --2- 172.21.15.87:0/3865126311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f047406e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04740c31a0 tx=0x7f046c02bed0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.427+0000 7f047b50e640 1 -- 172.21.15.87:0/3865126311 shutdown_connections 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.427+0000 7f047b50e640 1 --2- 172.21.15.87:0/3865126311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f047406e9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.427+0000 7f047b50e640 1 --2- 172.21.15.87:0/3865126311 >> v2:172.21.15.116:3300/0 conn(0x7f0474151740 0x7f04740c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.427+0000 7f047b50e640 1 -- 172.21.15.87:0/3865126311 >> 172.21.15.87:0/3865126311 conn(0x7f047406b6c0 msgr2=0x7f047406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.427+0000 7f047b50e640 1 -- 172.21.15.87:0/3865126311 shutdown_connections 2024-01-20T08:31:46.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.427+0000 7f047b50e640 1 -- 172.21.15.87:0/3865126311 wait complete. 2024-01-20T08:31:46.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.428+0000 7f0c24d53640 1 -- 172.21.15.87:0/3596907494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c200c9300 msgr2=0x7f0c201ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.428+0000 7f0c24d53640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c200c9300 0x7f0c201ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.428+0000 7f0c24d53640 1 -- 172.21.15.87:0/3596907494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c200700e0 msgr2=0x7f0c201e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.428+0000 7f0c24d53640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c200700e0 0x7f0c201e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.428+0000 7f0c24d53640 1 -- 172.21.15.87:0/3596907494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14009400 con 0x7f0c20153b50 2024-01-20T08:31:46.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.429+0000 7f047b50e640 1 Processor -- start 2024-01-20T08:31:46.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.429+0000 7f047b50e640 1 -- start start 2024-01-20T08:31:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.430+0000 7f047b50e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04740c5d30 0x7f04741549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.430+0000 7f047b50e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0474151740 0x7f0474147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.430+0000 7f047b50e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f0474152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.430+0000 7f047b50e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04741528b0 con 0x7f0474157370 2024-01-20T08:31:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.430+0000 7f047b50e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474151cc0 con 0x7f04740c5d30 2024-01-20T08:31:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.430+0000 7f047b50e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04741f3f00 con 0x7f0474151740 2024-01-20T08:31:46.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.432+0000 7f0c24d53640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 0x7f0c201ef7f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c9f0 tx=0x7f0c1800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.434+0000 7f0479aab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f0474152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.437+0000 7f0479aab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f0474152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45646/0 (socket says 172.21.15.87:45646) 2024-01-20T08:31:46.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.437+0000 7f0479aab640 1 -- 172.21.15.87:0/1499847487 learned_addr learned my addr 172.21.15.87:0/1499847487 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.437+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800ab70 con 0x7f0c20153b50 2024-01-20T08:31:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.437+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800acd0 con 0x7f0c20153b50 2024-01-20T08:31:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.437+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18012600 con 0x7f0c20153b50 2024-01-20T08:31:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.437+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c201f1ed0 con 0x7f0c20153b50 2024-01-20T08:31:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.438+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c201f2490 con 0x7f0c20153b50 2024-01-20T08:31:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.439+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c201f2680 con 0x7f0c20153b50 2024-01-20T08:31:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.465+0000 7f04792aa640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04740c5d30 0x7f04741549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.470+0000 7f04792aa640 1 -- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0474151740 msgr2=0x7f0474147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.470+0000 7f04792aa640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0474151740 0x7f0474147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.470+0000 7f04792aa640 1 -- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 msgr2=0x7f0474152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.470+0000 7f04792aa640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f0474152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.470+0000 7f04792aa640 1 -- 172.21.15.87:0/1499847487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c009a70 con 0x7f04740c5d30 2024-01-20T08:31:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.471+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0c18007070 con 0x7f0c20153b50 2024-01-20T08:31:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.472+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f678c06bc30 con 0x7f678c151640 2024-01-20T08:31:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.472+0000 7f0c1cff9640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c0c06af20 0x7f0c0c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.472+0000 7f0c1ffff640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c0c06af20 0x7f0c0c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.473+0000 7f0c1ffff640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c0c06af20 0x7f0c0c06d3e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c10003d10 tx=0x7f0c100079b0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.473+0000 7f04792aa640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04740c5d30 0x7f04741549d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f047000b3f0 tx=0x7f047000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.477+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470011830 con 0x7f04740c5d30 2024-01-20T08:31:46.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.477+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0470011990 con 0x7f04740c5d30 2024-01-20T08:31:46.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.477+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470011d40 con 0x7f04740c5d30 2024-01-20T08:31:46.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.478+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0474151f10 con 0x7f04740c5d30 2024-01-20T08:31:46.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.478+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04741524d0 con 0x7f04740c5d30 2024-01-20T08:31:46.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.479+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0474074290 con 0x7f04740c5d30 2024-01-20T08:31:46.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-20T08:31:46.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.502+0000 7fa1149a9640 1 Processor -- start 2024-01-20T08:31:46.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.502+0000 7fa1149a9640 1 -- start start 2024-01-20T08:31:46.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.503+0000 7fa1149a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa1100c8f40 0x7fa1100ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.503+0000 7fa1149a9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa1100c23f0 0x7fa1100cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.503+0000 7fa1149a9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1100c7d20 con 0x7fa110151950 2024-01-20T08:31:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.503+0000 7fa1149a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100c85b0 con 0x7fa1100c8f40 2024-01-20T08:31:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.503+0000 7fa1149a9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa110158490 con 0x7fa1100c23f0 2024-01-20T08:31:46.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.562+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9054079dc0 con 0x7f90580c9e00 2024-01-20T08:31:46.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.572+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9054076100 con 0x7f90580c9e00 2024-01-20T08:31:46.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.602+0000 7f1553c23640 1 Processor -- start 2024-01-20T08:31:46.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.602+0000 7f1553c23640 1 -- start start 2024-01-20T08:31:46.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.603+0000 7f1553c23640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f154c15a2b0 0x7f154c165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.603+0000 7f1553c23640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0cc3c0 con 0x7f154c0c9e00 2024-01-20T08:31:46.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.603+0000 7f1553c23640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0ca780 con 0x7f154c153a20 2024-01-20T08:31:46.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.603+0000 7f1553c23640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154c0c4670 con 0x7f154c15a2b0 2024-01-20T08:31:46.684 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-20T08:31:46.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.731+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f047001a400 con 0x7f04740c5d30 2024-01-20T08:31:46.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.731+0000 7f0469ffb640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f045806af20 0x7f045806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.731+0000 7f0478aa9640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f045806af20 0x7f045806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.731+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0c18058520 con 0x7f0c20153b50 2024-01-20T08:31:46.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.732+0000 7f0478aa9640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f045806af20 0x7f045806d3e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0464005880 tx=0x7f0464007aa0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.732+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0c18054860 con 0x7f0c20153b50 2024-01-20T08:31:46.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.734+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0470058f60 con 0x7f04740c5d30 2024-01-20T08:31:46.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.738+0000 7f15521c0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f154c15a2b0 0x7f154c165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.740+0000 7f15521c0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f154c15a2b0 0x7f154c165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45668/0 (socket says 172.21.15.87:45668) 2024-01-20T08:31:46.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.740+0000 7f15521c0640 1 -- 172.21.15.87:0/1959602616 learned_addr learned my addr 172.21.15.87:0/1959602616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.744+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0470054ba0 con 0x7f04740c5d30 2024-01-20T08:31:46.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.745+0000 7fa10ed9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa1100c23f0 0x7fa1100cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.747+0000 7fa10e59c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa110151950 0x7fa1100c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:36418/0 (socket says 172.21.15.87:36418) 2024-01-20T08:31:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.747+0000 7fa10e59c640 1 -- 172.21.15.87:0/773808553 learned_addr learned my addr 172.21.15.87:0/773808553 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.754+0000 7fa10ed9d640 1 -- 172.21.15.87:0/773808553 >> v2:172.21.15.87:3300/0 conn(0x7fa1100c8f40 msgr2=0x7fa1100ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.754+0000 7fa10ed9d640 1 --2- 172.21.15.87:0/773808553 >> v2:172.21.15.87:3300/0 conn(0x7fa1100c8f40 0x7fa1100ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.754+0000 7fa10ed9d640 1 -- 172.21.15.87:0/773808553 >> v1:172.21.15.116:6789/0 conn(0x7fa110151950 legacy=0x7fa1100c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.754+0000 7fa10ed9d640 1 -- 172.21.15.87:0/773808553 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100c8000 con 0x7fa1100c23f0 2024-01-20T08:31:46.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.757+0000 7fa10ed9d640 1 --2- 172.21.15.87:0/773808553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1100cd260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa110070830 tx=0x7fa10002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0993604d33202 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.761+0000 7fa10d59a640 1 -- 172.21.15.87:0/773808553 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100030610 con 0x7fa1100c23f0 2024-01-20T08:31:46.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.761+0000 7fa10d59a640 1 -- 172.21.15.87:0/773808553 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa100030770 con 0x7fa1100c23f0 2024-01-20T08:31:46.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.761+0000 7fa10d59a640 1 -- 172.21.15.87:0/773808553 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100030b00 con 0x7fa1100c23f0 2024-01-20T08:31:46.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.762+0000 7fa1149a9640 1 -- 172.21.15.87:0/773808553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 msgr2=0x7fa1100cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.762+0000 7fa1149a9640 1 --2- 172.21.15.87:0/773808553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1100cd260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa110070830 tx=0x7fa10002b960 comp rx=0 tx=0).stop 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.762+0000 7fa1149a9640 1 -- 172.21.15.87:0/773808553 shutdown_connections 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.762+0000 7fa1149a9640 1 --2- 172.21.15.87:0/773808553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1100cd260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.762+0000 7fa1149a9640 1 --2- 172.21.15.87:0/773808553 >> v2:172.21.15.87:3300/0 conn(0x7fa1100c8f40 0x7fa1100ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.762+0000 7fa1149a9640 1 -- 172.21.15.87:0/773808553 >> 172.21.15.87:0/773808553 conn(0x7fa11006a5b0 msgr2=0x7fa11006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.763+0000 7fa1149a9640 1 -- 172.21.15.87:0/773808553 shutdown_connections 2024-01-20T08:31:46.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.763+0000 7fa1149a9640 1 -- 172.21.15.87:0/773808553 wait complete. 2024-01-20T08:31:46.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.765+0000 7fa1149a9640 1 Processor -- start 2024-01-20T08:31:46.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.765+0000 7fa1149a9640 1 -- start start 2024-01-20T08:31:46.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.766+0000 7fa1149a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1101553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.766+0000 7fa1149a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1100c8f40 0x7fa1100cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.766+0000 7fa1149a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa110151950 0x7fa110147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.766+0000 7fa1149a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101ee320 con 0x7fa1100c8f40 2024-01-20T08:31:46.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.766+0000 7fa1149a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cc310 con 0x7fa1100c23f0 2024-01-20T08:31:46.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.766+0000 7fa1149a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cb940 con 0x7fa110151950 2024-01-20T08:31:46.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.767+0000 7fa10e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1101553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.767+0000 7fa10ed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa110151950 0x7fa110147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.767+0000 7fa10e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1101553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46546/0 (socket says 172.21.15.87:46546) 2024-01-20T08:31:46.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.767+0000 7fa10e59c640 1 -- 172.21.15.87:0/3115535402 learned_addr learned my addr 172.21.15.87:0/3115535402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.770+0000 7fa10e59c640 1 -- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa110151950 msgr2=0x7fa110147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.770+0000 7fa10e59c640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa110151950 0x7fa110147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.770+0000 7fa10e59c640 1 -- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1100c8f40 msgr2=0x7fa1100cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.770+0000 7fa10e59c640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1100c8f40 0x7fa1100cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.770+0000 7fa10e59c640 1 -- 172.21.15.87:0/3115535402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009400 con 0x7fa1100c23f0 2024-01-20T08:31:46.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.773+0000 7fa10e59c640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1101553f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa10800caa0 tx=0x7fa10800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.776+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800ac00 con 0x7fa1100c23f0 2024-01-20T08:31:46.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.776+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10800ad60 con 0x7fa1100c23f0 2024-01-20T08:31:46.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.776+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1101edd80 con 0x7fa1100c23f0 2024-01-20T08:31:46.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.776+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108012620 con 0x7fa1100c23f0 2024-01-20T08:31:46.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.777+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1100cb010 con 0x7fa1100c23f0 2024-01-20T08:31:46.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.778+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa110074290 con 0x7fa1100c23f0 2024-01-20T08:31:46.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.778+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9058154410 con 0x7f90580c9e00 2024-01-20T08:31:46.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.782+0000 7f904e7fc640 1 -- 172.21.15.87:0/4037944065 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9058154410 con 0x7f90580c9e00 2024-01-20T08:31:46.785 INFO:teuthology.orchestra.run.smithi087.stdout:60129542159 2024-01-20T08:31:46.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f903c06af20 msgr2=0x7f903c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f903c06af20 0x7f903c06d3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9058154050 tx=0x7f9048009040 comp rx=0 tx=0).stop 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 msgr2=0x7f9058156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 0x7f9058156980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9054034040 tx=0x7f90540046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 shutdown_connections 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f903c06af20 0x7f903c06d3e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9058153a20 0x7f90581efb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90580cefd0 0x7f90581ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 --2- 172.21.15.87:0/4037944065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90580c9e00 0x7f9058156980 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9054034040 tx=0x7f90540046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 >> 172.21.15.87:0/4037944065 conn(0x7f905808af90 msgr2=0x7f90580cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 shutdown_connections 2024-01-20T08:31:46.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.784+0000 7f905ffb6640 1 -- 172.21.15.87:0/4037944065 wait complete. 2024-01-20T08:31:46.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.785+0000 7f15521c0640 1 -- 172.21.15.87:0/1959602616 >> v1:172.21.15.116:6789/0 conn(0x7f154c153a20 legacy=0x7f154c0c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.785+0000 7f15521c0640 1 -- 172.21.15.87:0/1959602616 >> v1:172.21.15.87:6789/0 conn(0x7f154c0c9e00 legacy=0x7f154c0cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.785+0000 7f15521c0640 1 -- 172.21.15.87:0/1959602616 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0cb4d0 con 0x7f154c15a2b0 2024-01-20T08:31:46.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.792+0000 7f15521c0640 1 --2- 172.21.15.87:0/1959602616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c15a2b0 0x7f154c165120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1544009e50 tx=0x7f154402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0f739ef8a31fa1 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542159 got 60129542159 for osd.5 2024-01-20T08:31:46.799 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:46.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.797+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa108007070 con 0x7fa1100c23f0 2024-01-20T08:31:46.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.798+0000 7fa0feffd640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa0f006af50 0x7fa0f006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.798+0000 7fa10dd9b640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa0f006af50 0x7fa0f006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.799+0000 7fa10dd9b640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa0f006af50 0x7fa0f006d410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa11014b550 tx=0x7fa104006c90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.803+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fa108058610 con 0x7fa1100c23f0 2024-01-20T08:31:46.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.806+0000 7f7af829b640 1 Processor -- start 2024-01-20T08:31:46.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.806+0000 7f7af829b640 1 -- start start 2024-01-20T08:31:46.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.806+0000 7f67737fe640 1 -- 172.21.15.87:0/1415389879 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6780021090 con 0x7f678c151640 2024-01-20T08:31:46.809 INFO:teuthology.orchestra.run.smithi087.stdout:55834574862 2024-01-20T08:31:46.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af829b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7af0153650 0x7f7af0159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af829b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 0x7f7af00725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af829b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7af00c67e0 0x7f7af006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af829b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af015a510 con 0x7f7af0153650 2024-01-20T08:31:46.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af829b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af0072b30 con 0x7f7af00c67e0 2024-01-20T08:31:46.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af829b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af006bcb0 con 0x7f7af0153a30 2024-01-20T08:31:46.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.807+0000 7f7af5836640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 0x7f7af00725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f7af5836640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 0x7f7af00725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47340/0 (socket says 172.21.15.87:47340) 2024-01-20T08:31:46.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f7af5836640 1 -- 172.21.15.87:0/3607440082 learned_addr learned my addr 172.21.15.87:0/3607440082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f15509bd640 1 -- 172.21.15.87:0/1959602616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154402ca80 con 0x7f154c15a2b0 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f15509bd640 1 -- 172.21.15.87:0/1959602616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154402cbe0 con 0x7f154c15a2b0 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f15509bd640 1 -- 172.21.15.87:0/1959602616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440343f0 con 0x7f154c15a2b0 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f1553c23640 1 -- 172.21.15.87:0/1959602616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c15a2b0 msgr2=0x7f154c165120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f1553c23640 1 --2- 172.21.15.87:0/1959602616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c15a2b0 0x7f154c165120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1544009e50 tx=0x7f154402bee0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f1553c23640 1 -- 172.21.15.87:0/1959602616 shutdown_connections 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.808+0000 7f1553c23640 1 --2- 172.21.15.87:0/1959602616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c15a2b0 0x7f154c165120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f1553c23640 1 -- 172.21.15.87:0/1959602616 >> 172.21.15.87:0/1959602616 conn(0x7f154c08af90 msgr2=0x7f154c08b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f1553c23640 1 -- 172.21.15.87:0/1959602616 shutdown_connections 2024-01-20T08:31:46.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f1553c23640 1 -- 172.21.15.87:0/1959602616 wait complete. 2024-01-20T08:31:46.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f677806aed0 msgr2=0x7f677806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f677806aed0 0x7f677806d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f677c036000 tx=0x7f677c030000 comp rx=0 tx=0).stop 2024-01-20T08:31:46.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 msgr2=0x7f678c152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 0x7f678c152370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f678000e840 tx=0x7f678000ed00 comp rx=0 tx=0).stop 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 shutdown_connections 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f677806aed0 0x7f677806d390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f678c151a20 0x7f678c1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f678c151640 0x7f678c152370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 --2- 172.21.15.87:0/1415389879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678c0c5a90 0x7f678c0c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 >> 172.21.15.87:0/1415389879 conn(0x7f678c0723c0 msgr2=0x7f678c06fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 shutdown_connections 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.809+0000 7f67912da640 1 -- 172.21.15.87:0/1415389879 wait complete. 2024-01-20T08:31:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f1553c23640 1 Processor -- start 2024-01-20T08:31:46.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f1553c23640 1 -- start start 2024-01-20T08:31:46.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f7af5836640 1 -- 172.21.15.87:0/3607440082 >> v2:172.21.15.116:3300/0 conn(0x7f7af00c67e0 msgr2=0x7f7af006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f7af5836640 1 --2- 172.21.15.87:0/3607440082 >> v2:172.21.15.116:3300/0 conn(0x7f7af00c67e0 0x7f7af006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f7af5836640 1 -- 172.21.15.87:0/3607440082 >> v2:172.21.15.87:3300/0 conn(0x7f7af0153650 msgr2=0x7f7af0159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:46.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f7af5836640 1 --2- 172.21.15.87:0/3607440082 >> v2:172.21.15.87:3300/0 conn(0x7f7af0153650 0x7f7af0159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.810+0000 7f7af5836640 1 -- 172.21.15.87:0/3607440082 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af006bf90 con 0x7f7af0153a30 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.811+0000 7f1553c23640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154c0c9e00 0x7f154c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.812+0000 7f1553c23640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c153a20 0x7f154c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.812+0000 7f1553c23640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 0x7f154c1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.812+0000 7f1553c23640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0cf850 con 0x7f154c153a20 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.812+0000 7f1553c23640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c154490 con 0x7f154c0c9e00 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.812+0000 7f1553c23640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0cfaf0 con 0x7f154c15a2b0 2024-01-20T08:31:46.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.812+0000 7f7af5836640 1 --2- 172.21.15.87:0/3607440082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 0x7f7af00725f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7af0066e80 tx=0x7f7ae002b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b50530abfb868ea server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.815+0000 7f15521c0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 0x7f154c1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.816+0000 7f15521c0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 0x7f154c1437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47346/0 (socket says 172.21.15.87:47346) 2024-01-20T08:31:46.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.816+0000 7f15521c0640 1 -- 172.21.15.87:0/1445923763 learned_addr learned my addr 172.21.15.87:0/1445923763 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.817+0000 7f7af5035640 1 -- 172.21.15.87:0/3607440082 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0030860 con 0x7f7af0153a30 2024-01-20T08:31:46.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.817+0000 7f7af5035640 1 -- 172.21.15.87:0/3607440082 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae00309c0 con 0x7f7af0153a30 2024-01-20T08:31:46.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.817+0000 7f7af5035640 1 -- 172.21.15.87:0/3607440082 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0030d50 con 0x7f7af0153a30 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 -- 172.21.15.87:0/3607440082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 msgr2=0x7f7af00725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 --2- 172.21.15.87:0/3607440082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 0x7f7af00725f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7af0066e80 tx=0x7f7ae002b9a0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 -- 172.21.15.87:0/3607440082 shutdown_connections 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 --2- 172.21.15.87:0/3607440082 >> v2:172.21.15.116:3300/0 conn(0x7f7af00c67e0 0x7f7af006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 --2- 172.21.15.87:0/3607440082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af0153a30 0x7f7af00725f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 --2- 172.21.15.87:0/3607440082 >> v2:172.21.15.87:3300/0 conn(0x7f7af0153650 0x7f7af0159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 -- 172.21.15.87:0/3607440082 >> 172.21.15.87:0/3607440082 conn(0x7f7af00cb3d0 msgr2=0x7f7af00cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 -- 172.21.15.87:0/3607440082 shutdown_connections 2024-01-20T08:31:46.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.818+0000 7f7af829b640 1 -- 172.21.15.87:0/3607440082 wait complete. 2024-01-20T08:31:46.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f7af829b640 1 Processor -- start 2024-01-20T08:31:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f15521c0640 1 -- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154c0c9e00 msgr2=0x7f154c156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f15521c0640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154c0c9e00 0x7f154c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f15521c0640 1 -- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c153a20 msgr2=0x7f154c15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f15521c0640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c153a20 0x7f154c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f15521c0640 1 -- 172.21.15.87:0/1445923763 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544009b30 con 0x7f154c15a2b0 2024-01-20T08:31:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.820+0000 7f7af829b640 1 -- start start 2024-01-20T08:31:46.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.821+0000 7f7af829b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 0x7f7af01f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.821+0000 7f7af829b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7af0153650 0x7f7af00ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.821+0000 7f7af829b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7af0153a30 0x7f7af01e1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.821+0000 7f7af829b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01f3a10 con 0x7f7af0153650 2024-01-20T08:31:46.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.821+0000 7f7af829b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00caea0 con 0x7f7af0153a30 2024-01-20T08:31:46.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.821+0000 7f7af829b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00ca990 con 0x7f7af00c67e0 2024-01-20T08:31:46.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.824+0000 7f15521c0640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 0x7f154c1437d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1544036040 tx=0x7f1544004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.824+0000 7f7af6037640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 0x7f7af01f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.7 2024-01-20T08:31:46.828 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:46.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.827+0000 7f7af6037640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 0x7f7af01f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47352/0 (socket says 172.21.15.87:47352) 2024-01-20T08:31:46.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.827+0000 7f7af6037640 1 -- 172.21.15.87:0/792855108 learned_addr learned my addr 172.21.15.87:0/792855108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:46.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.828+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154403e450 con 0x7f154c15a2b0 2024-01-20T08:31:46.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.828+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154403e5b0 con 0x7f154c15a2b0 2024-01-20T08:31:46.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.828+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154403e960 con 0x7f154c15a2b0 2024-01-20T08:31:46.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.828+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c0cfdd0 con 0x7f154c15a2b0 2024-01-20T08:31:46.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.829+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c1524f0 con 0x7f154c15a2b0 2024-01-20T08:31:46.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.830+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c156a00 con 0x7f154c15a2b0 2024-01-20T08:31:46.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.831+0000 7f7af6037640 1 -- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7af0153a30 msgr2=0x7f7af01e1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.831+0000 7f7af6037640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7af0153a30 0x7f7af01e1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.831+0000 7f7af6037640 1 -- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7af0153650 msgr2=0x7f7af00ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.831+0000 7f7af6037640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7af0153650 0x7f7af00ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.831+0000 7f7af6037640 1 -- 172.21.15.87:0/792855108 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00094d0 con 0x7f7af00c67e0 2024-01-20T08:31:46.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.834+0000 7f7af6037640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 0x7f7af01f34d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b4a0 tx=0x7f7aec00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.858+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f154402cd50 con 0x7f154c15a2b0 2024-01-20T08:31:46.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.859+0000 7f15427fc640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f153006af20 0x7f153006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:46.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.859+0000 7f15519bf640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f153006af20 0x7f153006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:46.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.860+0000 7f15519bf640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f153006af20 0x7f153006d3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1548000f60 tx=0x7f15480079d0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:46.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.910+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f154407bc40 con 0x7f154c15a2b0 2024-01-20T08:31:46.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.938+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0c20074290 con 0x7f0c20153b50 2024-01-20T08:31:46.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.942+0000 7f0c1cff9640 1 -- 172.21.15.87:0/3596907494 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c18017090 con 0x7f0c20153b50 2024-01-20T08:31:46.945 INFO:teuthology.orchestra.run.smithi087.stdout:60129542159 2024-01-20T08:31:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c0c06af20 msgr2=0x7f0c0c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c0c06af20 0x7f0c0c06d3e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c10003d10 tx=0x7f0c100079b0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 msgr2=0x7f0c201ef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 0x7f0c201ef7f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c9f0 tx=0x7f0c1800ceb0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 shutdown_connections 2024-01-20T08:31:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c0c06af20 0x7f0c0c06d3e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c20153b50 0x7f0c201ef7f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c200c9300 0x7f0c201ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 --2- 172.21.15.87:0/3596907494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c200700e0 0x7f0c201e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:46.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 >> 172.21.15.87:0/3596907494 conn(0x7f0c20072810 msgr2=0x7f0c200c21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:46.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 shutdown_connections 2024-01-20T08:31:46.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.944+0000 7f0c267b6640 1 -- 172.21.15.87:0/3596907494 wait complete. 2024-01-20T08:31:46.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:46.949+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f04740c4010 con 0x7f04740c5d30 2024-01-20T08:31:46.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542159 got 60129542159 for osd.2 2024-01-20T08:31:46.958 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:47.040 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-01-20T08:31:47.037+0000 f5d6640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-01-20T08:31:47.040 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-01-20T08:31:47.037+0000 f5d6640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-01-20T08:31:47.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.046+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa108054950 con 0x7fa1100c23f0 2024-01-20T08:31:47.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.119+0000 7f0469ffb640 1 -- 172.21.15.87:0/1499847487 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0470015020 con 0x7f04740c5d30 2024-01-20T08:31:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:64424509456 2024-01-20T08:31:47.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f045806af20 msgr2=0x7f045806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f045806af20 0x7f045806d3e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0464005880 tx=0x7f0464007aa0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04740c5d30 msgr2=0x7f04741549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04740c5d30 0x7f04741549d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f047000b3f0 tx=0x7f047000b8b0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 shutdown_connections 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f045806af20 0x7f045806d3e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0474157370 0x7f0474152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0474151740 0x7f0474147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 --2- 172.21.15.87:0/1499847487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04740c5d30 0x7f04741549d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 >> 172.21.15.87:0/1499847487 conn(0x7f047406b6c0 msgr2=0x7f047406e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 shutdown_connections 2024-01-20T08:31:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.121+0000 7f047b50e640 1 -- 172.21.15.87:0/1499847487 wait complete. 2024-01-20T08:31:47.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509457 got 64424509456 for osd.0 2024-01-20T08:31:47.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.159+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1544077880 con 0x7f154c15a2b0 2024-01-20T08:31:47.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.161+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec010610 con 0x7f7af00c67e0 2024-01-20T08:31:47.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.162+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec010770 con 0x7f7af00c67e0 2024-01-20T08:31:47.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.162+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af01ef8a0 con 0x7f7af00c67e0 2024-01-20T08:31:47.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.162+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec010b20 con 0x7f7af00c67e0 2024-01-20T08:31:47.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.162+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01efe60 con 0x7f7af00c67e0 2024-01-20T08:31:47.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.162+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af00c31e0 con 0x7f7af00c67e0 2024-01-20T08:31:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.247+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa11014b7a0 con 0x7fa1100c23f0 2024-01-20T08:31:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.251+0000 7fa0feffd640 1 -- 172.21.15.87:0/3115535402 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa108017020 con 0x7fa1100c23f0 2024-01-20T08:31:47.254 INFO:teuthology.orchestra.run.smithi087.stdout:55834574862 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.253+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa0f006af50 msgr2=0x7fa0f006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.253+0000 7fa1149a9640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa0f006af50 0x7fa0f006d410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa11014b550 tx=0x7fa104006c90 comp rx=0 tx=0).stop 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.253+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 msgr2=0x7fa1101553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.253+0000 7fa1149a9640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1101553f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa10800caa0 tx=0x7fa10800cf60 comp rx=0 tx=0).stop 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 shutdown_connections 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa0f006af50 0x7fa0f006d410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa110151950 0x7fa110147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1100c8f40 0x7fa1100cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 --2- 172.21.15.87:0/3115535402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1100c23f0 0x7fa1101553f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 >> 172.21.15.87:0/3115535402 conn(0x7fa11006a5b0 msgr2=0x7fa1100c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:47.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 shutdown_connections 2024-01-20T08:31:47.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.254+0000 7fa1149a9640 1 -- 172.21.15.87:0/3115535402 wait complete. 2024-01-20T08:31:47.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.3 2024-01-20T08:31:47.268 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:47.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.359+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f154c074290 con 0x7f154c15a2b0 2024-01-20T08:31:47.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.404+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7aec010c80 con 0x7f7af00c67e0 2024-01-20T08:31:47.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.405+0000 7f7ae6ffd640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7ad406af20 0x7f7ad406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:47.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.405+0000 7f7af5836640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7ad406af20 0x7f7ad406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:47.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.405+0000 7f7af5836640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7ad406af20 0x7f7ad406d3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0036000 tx=0x7f7ae003b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:47.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.408+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7aec058920 con 0x7f7af00c67e0 2024-01-20T08:31:47.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.417+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7aec054560 con 0x7f7af00c67e0 2024-01-20T08:31:47.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.425+0000 7f15427fc640 1 -- 172.21.15.87:0/1445923763 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1544043020 con 0x7f154c15a2b0 2024-01-20T08:31:47.428 INFO:teuthology.orchestra.run.smithi087.stdout:64424509457 2024-01-20T08:31:47.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f153006af20 msgr2=0x7f153006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f153006af20 0x7f153006d3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1548000f60 tx=0x7f15480079d0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 msgr2=0x7f154c1437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 0x7f154c1437d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1544036040 tx=0x7f1544004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:47.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 shutdown_connections 2024-01-20T08:31:47.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f153006af20 0x7f153006d3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f154c15a2b0 0x7f154c1437d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f154c153a20 0x7f154c15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 --2- 172.21.15.87:0/1445923763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154c0c9e00 0x7f154c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 >> 172.21.15.87:0/1445923763 conn(0x7f154c08af90 msgr2=0x7f154c158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:47.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 shutdown_connections 2024-01-20T08:31:47.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.428+0000 7f1553c23640 1 -- 172.21.15.87:0/1445923763 wait complete. 2024-01-20T08:31:47.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509457 got 64424509457 for osd.6 2024-01-20T08:31:47.442 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:47.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.597+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7af00cc090 con 0x7f7af00c67e0 2024-01-20T08:31:47.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.603+0000 7f7ae6ffd640 1 -- 172.21.15.87:0/792855108 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7aec01e070 con 0x7f7af00c67e0 2024-01-20T08:31:47.606 INFO:teuthology.orchestra.run.smithi087.stdout:60129542160 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7ad406af20 msgr2=0x7f7ad406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7ad406af20 0x7f7ad406d3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0036000 tx=0x7f7ae003b040 comp rx=0 tx=0).stop 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 msgr2=0x7f7af01f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 0x7f7af01f34d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b4a0 tx=0x7f7aec00b960 comp rx=0 tx=0).stop 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 shutdown_connections 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7ad406af20 0x7f7ad406d3e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7af0153a30 0x7f7af01e1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7af0153650 0x7f7af00ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 --2- 172.21.15.87:0/792855108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7af00c67e0 0x7f7af01f34d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:47.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 >> 172.21.15.87:0/792855108 conn(0x7f7af00cb3d0 msgr2=0x7f7af01501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:47.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 shutdown_connections 2024-01-20T08:31:47.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:47.605+0000 7f7af829b640 1 -- 172.21.15.87:0/792855108 wait complete. 2024-01-20T08:31:47.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542160 got 60129542160 for osd.1 2024-01-20T08:31:47.619 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:48.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-20T08:31:48.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.237+0000 7fba9f231640 1 Processor -- start 2024-01-20T08:31:48.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.237+0000 7fba9f231640 1 -- start start 2024-01-20T08:31:48.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.237+0000 7fba9f231640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba98153800 0x7fba980c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.237+0000 7fba9f231640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fba9806f670 0x7fba98159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.238+0000 7fba9f231640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9815cb70 con 0x7fba980c9f10 2024-01-20T08:31:48.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.238+0000 7fba9f231640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980c2730 con 0x7fba9806f670 2024-01-20T08:31:48.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.238+0000 7fba9f231640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980c9430 con 0x7fba98153800 2024-01-20T08:31:48.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.239+0000 7fba97fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fba980c9f10 0x7fba9806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:36440/0 (socket says 172.21.15.87:36440) 2024-01-20T08:31:48.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.239+0000 7fba97fff640 1 -- 172.21.15.87:0/3125000042 learned_addr learned my addr 172.21.15.87:0/3125000042 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:48.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.239+0000 7fba9d7ce640 1 --2- 172.21.15.87:0/3125000042 >> v2:172.21.15.116:3300/0 conn(0x7fba9806f670 0x7fba98159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.243+0000 7fba9d7ce640 1 -- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba98153800 msgr2=0x7fba980c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:48.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.243+0000 7fba9d7ce640 1 --2- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba98153800 0x7fba980c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.243+0000 7fba9d7ce640 1 -- 172.21.15.87:0/3125000042 >> v1:172.21.15.116:6789/0 conn(0x7fba980c9f10 legacy=0x7fba9806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.243+0000 7fba9d7ce640 1 -- 172.21.15.87:0/3125000042 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9815d9d0 con 0x7fba9806f670 2024-01-20T08:31:48.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.248+0000 7fba9d7ce640 1 --2- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba98159710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba90009df0 tx=0x7fba9002be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9fcfa1b171ed85c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.252+0000 7fba977fe640 1 -- 172.21.15.87:0/3125000042 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9002cc20 con 0x7fba9806f670 2024-01-20T08:31:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.252+0000 7fba977fe640 1 -- 172.21.15.87:0/3125000042 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9002cd80 con 0x7fba9806f670 2024-01-20T08:31:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.252+0000 7fba977fe640 1 -- 172.21.15.87:0/3125000042 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90034660 con 0x7fba9806f670 2024-01-20T08:31:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 -- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 msgr2=0x7fba98159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 --2- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba98159710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba90009df0 tx=0x7fba9002be70 comp rx=0 tx=0).stop 2024-01-20T08:31:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 -- 172.21.15.87:0/3125000042 shutdown_connections 2024-01-20T08:31:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 --2- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba98159710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 --2- 172.21.15.87:0/3125000042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba98153800 0x7fba980c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 -- 172.21.15.87:0/3125000042 >> 172.21.15.87:0/3125000042 conn(0x7fba98070de0 msgr2=0x7fba980711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 -- 172.21.15.87:0/3125000042 shutdown_connections 2024-01-20T08:31:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.253+0000 7fba9f231640 1 -- 172.21.15.87:0/3125000042 wait complete. 2024-01-20T08:31:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.254+0000 7fba9f231640 1 Processor -- start 2024-01-20T08:31:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.254+0000 7fba9f231640 1 -- start start 2024-01-20T08:31:48.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.255+0000 7fba9f231640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba981f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.255+0000 7fba9f231640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba980c9f10 0x7fba981e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.255+0000 7fba9f231640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba98153800 0x7fba981509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.255+0000 7fba9f231640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981ef680 con 0x7fba98153800 2024-01-20T08:31:48.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.255+0000 7fba9f231640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981f3b40 con 0x7fba9806f670 2024-01-20T08:31:48.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.255+0000 7fba9f231640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba981583f0 con 0x7fba980c9f10 2024-01-20T08:31:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.256+0000 7fba9cfcd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba981f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.256+0000 7fba97fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba980c9f10 0x7fba981e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.256+0000 7fba9cfcd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba981f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46594/0 (socket says 172.21.15.87:46594) 2024-01-20T08:31:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.256+0000 7fba9cfcd640 1 -- 172.21.15.87:0/3232429364 learned_addr learned my addr 172.21.15.87:0/3232429364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.259+0000 7fba9d7ce640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba98153800 0x7fba981509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.259+0000 7fba97fff640 1 -- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 msgr2=0x7fba981f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.259+0000 7fba97fff640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba981f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.259+0000 7fba97fff640 1 -- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba98153800 msgr2=0x7fba981509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.259+0000 7fba97fff640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba98153800 0x7fba981509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.259+0000 7fba97fff640 1 -- 172.21.15.87:0/3232429364 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009a70 con 0x7fba980c9f10 2024-01-20T08:31:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.263+0000 7fba97fff640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba980c9f10 0x7fba981e4ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fba84000f60 tx=0x7fba8400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.266+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84005ce0 con 0x7fba980c9f10 2024-01-20T08:31:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.267+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84005e40 con 0x7fba980c9f10 2024-01-20T08:31:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.267+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba981504e0 con 0x7fba980c9f10 2024-01-20T08:31:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.267+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84007310 con 0x7fba980c9f10 2024-01-20T08:31:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.267+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981efb30 con 0x7fba980c9f10 2024-01-20T08:31:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.268+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba98074290 con 0x7fba980c9f10 2024-01-20T08:31:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.506+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fba84007470 con 0x7fba980c9f10 2024-01-20T08:31:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.507+0000 7fba957fa640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fba7c06af50 0x7fba7c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.507+0000 7fba9cfcd640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fba7c06af50 0x7fba7c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.507+0000 7fba9cfcd640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fba7c06af50 0x7fba7c06d410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0099a0 tx=0x7fba8c006f90 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.513+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fba84007720 con 0x7fba980c9f10 2024-01-20T08:31:48.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.522+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fba84053ea0 con 0x7fba980c9f10 2024-01-20T08:31:48.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.695+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fba980c8650 con 0x7fba980c9f10 2024-01-20T08:31:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.699+0000 7fba957fa640 1 -- 172.21.15.87:0/3232429364 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba84017070 con 0x7fba980c9f10 2024-01-20T08:31:48.702 INFO:teuthology.orchestra.run.smithi087.stdout:64424509457 2024-01-20T08:31:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fba7c06af50 msgr2=0x7fba7c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fba7c06af50 0x7fba7c06d410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0099a0 tx=0x7fba8c006f90 comp rx=0 tx=0).stop 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba980c9f10 msgr2=0x7fba981e4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba980c9f10 0x7fba981e4ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fba84000f60 tx=0x7fba8400c800 comp rx=0 tx=0).stop 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 shutdown_connections 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fba7c06af50 0x7fba7c06d410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba98153800 0x7fba981509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba980c9f10 0x7fba981e4ea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 --2- 172.21.15.87:0/3232429364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba9806f670 0x7fba981f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 >> 172.21.15.87:0/3232429364 conn(0x7fba98070de0 msgr2=0x7fba98073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 shutdown_connections 2024-01-20T08:31:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.701+0000 7fba9f231640 1 -- 172.21.15.87:0/3232429364 wait complete. 2024-01-20T08:31:48.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509457 got 64424509457 for osd.0 2024-01-20T08:31:48.715 DEBUG:teuthology.parallel:result is None 2024-01-20T08:31:48.715 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-20T08:31:48.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:31:48.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.812+0000 7f936fbb0640 1 Processor -- start 2024-01-20T08:31:48.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.812+0000 7f936fbb0640 1 -- start start 2024-01-20T08:31:48.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.813+0000 7f936fbb0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f93680ca260 0x7f93680c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.813+0000 7f936fbb0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936815ba40 con 0x7f93680710d0 2024-01-20T08:31:48.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.813+0000 7f936fbb0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93681576e0 con 0x7f9368070cf0 2024-01-20T08:31:48.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.813+0000 7f936fbb0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93680c3bb0 con 0x7f93680ca260 2024-01-20T08:31:48.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.814+0000 7f936e14d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f93680ca260 0x7f93680c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.814+0000 7f936e14d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f93680ca260 0x7f93680c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45736/0 (socket says 172.21.15.87:45736) 2024-01-20T08:31:48.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.814+0000 7f936e14d640 1 -- 172.21.15.87:0/2097277551 learned_addr learned my addr 172.21.15.87:0/2097277551 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:48.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.819+0000 7f936e14d640 1 -- 172.21.15.87:0/2097277551 >> v1:172.21.15.116:6789/0 conn(0x7f9368070cf0 legacy=0x7f9368157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.819+0000 7f936e14d640 1 -- 172.21.15.87:0/2097277551 >> v1:172.21.15.87:6789/0 conn(0x7f93680710d0 legacy=0x7f9368151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.819+0000 7f936e14d640 1 -- 172.21.15.87:0/2097277551 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936815aec0 con 0x7f93680ca260 2024-01-20T08:31:48.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.821+0000 7f936e14d640 1 --2- 172.21.15.87:0/2097277551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93680c7680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9360009870 tx=0x7f936002bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d0860e5b0cdcd server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:48.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.825+0000 7f936c94a640 1 -- 172.21.15.87:0/2097277551 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936002d870 con 0x7f93680ca260 2024-01-20T08:31:48.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.825+0000 7f936c94a640 1 -- 172.21.15.87:0/2097277551 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936002d9d0 con 0x7f93680ca260 2024-01-20T08:31:48.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.825+0000 7f936c94a640 1 -- 172.21.15.87:0/2097277551 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936002dd60 con 0x7f93680ca260 2024-01-20T08:31:48.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 -- 172.21.15.87:0/2097277551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 msgr2=0x7f93680c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2097277551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93680c7680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9360009870 tx=0x7f936002bab0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 -- 172.21.15.87:0/2097277551 shutdown_connections 2024-01-20T08:31:48.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2097277551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93680c7680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 -- 172.21.15.87:0/2097277551 >> 172.21.15.87:0/2097277551 conn(0x7f9368066ac0 msgr2=0x7f936806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:48.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 -- 172.21.15.87:0/2097277551 shutdown_connections 2024-01-20T08:31:48.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.826+0000 7f936fbb0640 1 -- 172.21.15.87:0/2097277551 wait complete. 2024-01-20T08:31:48.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.827+0000 7f936fbb0640 1 Processor -- start 2024-01-20T08:31:48.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.827+0000 7f936fbb0640 1 -- start start 2024-01-20T08:31:48.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.828+0000 7f936fbb0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 0x7f9368151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.828+0000 7f936fbb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93681ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.828+0000 7f936fbb0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f93681ee5f0 0x7f93680cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.828+0000 7f936fbb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93680cb9e0 con 0x7f93680ca260 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.828+0000 7f936fbb0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93680cc3b0 con 0x7f93680710d0 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.828+0000 7f936fbb0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93680cab10 con 0x7f93681ee5f0 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.829+0000 7f936d14b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93681ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.829+0000 7f936d94c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 0x7f9368151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.829+0000 7f936e14d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f93681ee5f0 0x7f93680cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:48.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.829+0000 7f936d94c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 0x7f9368151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46604/0 (socket says 172.21.15.87:46604) 2024-01-20T08:31:48.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.829+0000 7f936d94c640 1 -- 172.21.15.87:0/2361613558 learned_addr learned my addr 172.21.15.87:0/2361613558 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:48.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.834+0000 7f936d94c640 1 -- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f93681ee5f0 msgr2=0x7f93680cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.834+0000 7f936d94c640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f93681ee5f0 0x7f93680cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.834+0000 7f936d94c640 1 -- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 msgr2=0x7f93681ed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:48.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.834+0000 7f936d94c640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93681ed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:48.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.834+0000 7f936d94c640 1 -- 172.21.15.87:0/2361613558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600094d0 con 0x7f93680710d0 2024-01-20T08:31:48.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.837+0000 7f936d94c640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 0x7f9368151ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f936400c730 tx=0x7f936400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:48.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.841+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364006d50 con 0x7f93680710d0 2024-01-20T08:31:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.841+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936400ce40 con 0x7f93680710d0 2024-01-20T08:31:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.841+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93680cadd0 con 0x7f93680710d0 2024-01-20T08:31:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.841+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364003bb0 con 0x7f93680710d0 2024-01-20T08:31:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.841+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93680cb390 con 0x7f93680710d0 2024-01-20T08:31:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:48.842+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93680c37c0 con 0x7f93680710d0 2024-01-20T08:31:49.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.091+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9364003d10 con 0x7f93680710d0 2024-01-20T08:31:49.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.091+0000 7f935e7fc640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f934c06aed0 0x7f934c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.091+0000 7f936d14b640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f934c06aed0 0x7f934c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.092+0000 7f936d14b640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f934c06aed0 0x7f934c06d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f936814b000 tx=0x7f9358009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:49.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.095+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9364058b60 con 0x7f93680710d0 2024-01-20T08:31:49.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.105+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9364058930 con 0x7f93680710d0 2024-01-20T08:31:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.327+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f936815b760 con 0x7f934c06aed0 2024-01-20T08:31:49.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.336+0000 7f935e7fc640 1 -- 172.21.15.87:0/2361613558 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345827 (secure 0 0 0) 0x7f93580014a0 con 0x7f934c06aed0 2024-01-20T08:31:49.342 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:49.344 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f934c06aed0 msgr2=0x7f934c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f934c06aed0 0x7f934c06d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f936814b000 tx=0x7f9358009040 comp rx=0 tx=0).stop 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 msgr2=0x7f9368151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 0x7f9368151ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f936400c730 tx=0x7f936400cbf0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 shutdown_connections 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f934c06aed0 0x7f934c06d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f93681ee5f0 0x7f93680cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93680ca260 0x7f93681ed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 --2- 172.21.15.87:0/2361613558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93680710d0 0x7f9368151ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 >> 172.21.15.87:0/2361613558 conn(0x7f9368070cf0 msgr2=0x7f9368159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:49.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 shutdown_connections 2024-01-20T08:31:49.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.339+0000 7f936fbb0640 1 -- 172.21.15.87:0/2361613558 wait complete. 2024-01-20T08:31:49.353 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-01-20T08:31:48.286014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599897,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":353,"num_write_kb":2479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":324,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219792,"kb_used_data":3624,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754928,"statfs":{"total":773094113280,"available":772869046272,"internally_reserved":0,"allocated":3710976,"data_stored":2133488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4286,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002149"},"pg_stats":[{"pgid":"2.1f","version":"27'13","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.812971+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:31:44.812971+0000","last_peered":"2024-01-20T08:31:44.812971+0000","last_clean":"2024-01-20T08:31:44.812971+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:31:44.812971+0000","last_undegraded":"2024-01-20T08:31:44.812971+0000","last_fullsized":"2024-01-20T08:31:44.812971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245064+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:31:43.245064+0000","last_peered":"2024-01-20T08:31:43.245064+0000","last_clean":"2024-01-20T08:31:43.245064+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:31:43.245064+0000","last_undegraded":"2024-01-20T08:31:43.245064+0000","last_fullsized":"2024-01-20T08:31:43.245064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.811554+0000","last_change":"2024-01-20T08:31:44.811554+0000","last_active":"2024-01-20T08:31:44.811554+0000","last_peered":"2024-01-20T08:31:44.811554+0000","last_clean":"2024-01-20T08:31:44.811554+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:31:44.811554+0000","last_undegraded":"2024-01-20T08:31:44.811554+0000","last_fullsized":"2024-01-20T08:31:44.811554+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:31:34.560783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245144+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:31:43.245144+0000","last_peered":"2024-01-20T08:31:43.245144+0000","last_clean":"2024-01-20T08:31:43.245144+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:31:43.245144+0000","last_undegraded":"2024-01-20T08:31:43.245144+0000","last_fullsized":"2024-01-20T08:31:43.245144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245051+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:31:43.245051+0000","last_peered":"2024-01-20T08:31:43.245051+0000","last_clean":"2024-01-20T08:31:43.245051+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:31:43.245051+0000","last_undegraded":"2024-01-20T08:31:43.245051+0000","last_fullsized":"2024-01-20T08:31:43.245051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244170+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:31:43.244170+0000","last_peered":"2024-01-20T08:31:43.244170+0000","last_clean":"2024-01-20T08:31:43.244170+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:31:43.244170+0000","last_undegraded":"2024-01-20T08:31:43.244170+0000","last_fullsized":"2024-01-20T08:31:43.244170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.893516+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:31:44.893516+0000","last_peered":"2024-01-20T08:31:44.893516+0000","last_clean":"2024-01-20T08:31:44.893516+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:31:44.893516+0000","last_undegraded":"2024-01-20T08:31:44.893516+0000","last_fullsized":"2024-01-20T08:31:44.893516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983392+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:31:42.983392+0000","last_peered":"2024-01-20T08:31:42.983392+0000","last_clean":"2024-01-20T08:31:42.983392+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:31:42.983392+0000","last_undegraded":"2024-01-20T08:31:42.983392+0000","last_fullsized":"2024-01-20T08:31:42.983392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244350+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:31:43.244350+0000","last_peered":"2024-01-20T08:31:43.244350+0000","last_clean":"2024-01-20T08:31:43.244350+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:31:43.244350+0000","last_undegraded":"2024-01-20T08:31:43.244350+0000","last_fullsized":"2024-01-20T08:31:43.244350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.893215+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:31:44.893215+0000","last_peered":"2024-01-20T08:31:44.893215+0000","last_clean":"2024-01-20T08:31:44.893215+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:31:44.893215+0000","last_undegraded":"2024-01-20T08:31:44.893215+0000","last_fullsized":"2024-01-20T08:31:44.893215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137012+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:31:17.137012+0000","last_peered":"2024-01-20T08:31:17.137012+0000","last_clean":"2024-01-20T08:31:17.137012+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:31:17.137012+0000","last_undegraded":"2024-01-20T08:31:17.137012+0000","last_fullsized":"2024-01-20T08:31:17.137012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244287+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:31:43.244287+0000","last_peered":"2024-01-20T08:31:43.244287+0000","last_clean":"2024-01-20T08:31:43.244287+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:31:43.244287+0000","last_undegraded":"2024-01-20T08:31:43.244287+0000","last_fullsized":"2024-01-20T08:31:43.244287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136470+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:31:17.136470+0000","last_peered":"2024-01-20T08:31:17.136470+0000","last_clean":"2024-01-20T08:31:17.136470+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:31:17.136470+0000","last_undegraded":"2024-01-20T08:31:17.136470+0000","last_fullsized":"2024-01-20T08:31:17.136470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999583+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:31:37.999583+0000","last_peered":"2024-01-20T08:31:37.999583+0000","last_clean":"2024-01-20T08:31:37.999583+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:31:37.999583+0000","last_undegraded":"2024-01-20T08:31:37.999583+0000","last_fullsized":"2024-01-20T08:31:37.999583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571539+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:31:19.571539+0000","last_peered":"2024-01-20T08:31:19.571539+0000","last_clean":"2024-01-20T08:31:19.571539+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:31:19.571539+0000","last_undegraded":"2024-01-20T08:31:19.571539+0000","last_fullsized":"2024-01-20T08:31:19.571539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981209+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:31:42.981209+0000","last_peered":"2024-01-20T08:31:42.981209+0000","last_clean":"2024-01-20T08:31:42.981209+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:31:42.981209+0000","last_undegraded":"2024-01-20T08:31:42.981209+0000","last_fullsized":"2024-01-20T08:31:42.981209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.209035+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:31:44.209035+0000","last_peered":"2024-01-20T08:31:44.209035+0000","last_clean":"2024-01-20T08:31:44.209035+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:31:44.209035+0000","last_undegraded":"2024-01-20T08:31:44.209035+0000","last_fullsized":"2024-01-20T08:31:44.209035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978645+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:31:42.978645+0000","last_peered":"2024-01-20T08:31:42.978645+0000","last_clean":"2024-01-20T08:31:42.978645+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:31:42.978645+0000","last_undegraded":"2024-01-20T08:31:42.978645+0000","last_fullsized":"2024-01-20T08:31:42.978645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984304+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:31:42.984304+0000","last_peered":"2024-01-20T08:31:42.984304+0000","last_clean":"2024-01-20T08:31:42.984304+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:31:42.984304+0000","last_undegraded":"2024-01-20T08:31:42.984304+0000","last_fullsized":"2024-01-20T08:31:42.984304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.812638+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:31:44.812638+0000","last_peered":"2024-01-20T08:31:44.812638+0000","last_clean":"2024-01-20T08:31:44.812638+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:31:44.812638+0000","last_undegraded":"2024-01-20T08:31:44.812638+0000","last_fullsized":"2024-01-20T08:31:44.812638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981846+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:31:42.981846+0000","last_peered":"2024-01-20T08:31:42.981846+0000","last_clean":"2024-01-20T08:31:42.981846+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:31:42.981846+0000","last_undegraded":"2024-01-20T08:31:42.981846+0000","last_fullsized":"2024-01-20T08:31:42.981846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136221+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:31:17.136221+0000","last_peered":"2024-01-20T08:31:17.136221+0000","last_clean":"2024-01-20T08:31:17.136221+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:31:17.136221+0000","last_undegraded":"2024-01-20T08:31:17.136221+0000","last_fullsized":"2024-01-20T08:31:17.136221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978331+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:31:42.978331+0000","last_peered":"2024-01-20T08:31:42.978331+0000","last_clean":"2024-01-20T08:31:42.978331+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:31:42.978331+0000","last_undegraded":"2024-01-20T08:31:42.978331+0000","last_fullsized":"2024-01-20T08:31:42.978331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244578+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:31:43.244578+0000","last_peered":"2024-01-20T08:31:43.244578+0000","last_clean":"2024-01-20T08:31:43.244578+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:31:43.244578+0000","last_undegraded":"2024-01-20T08:31:43.244578+0000","last_fullsized":"2024-01-20T08:31:43.244578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984212+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:31:42.984212+0000","last_peered":"2024-01-20T08:31:42.984212+0000","last_clean":"2024-01-20T08:31:42.984212+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:31:42.984212+0000","last_undegraded":"2024-01-20T08:31:42.984212+0000","last_fullsized":"2024-01-20T08:31:42.984212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'5","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603420+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:31:42.603420+0000","last_peered":"2024-01-20T08:31:42.603420+0000","last_clean":"2024-01-20T08:31:42.603420+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:31:42.603420+0000","last_undegraded":"2024-01-20T08:31:42.603420+0000","last_fullsized":"2024-01-20T08:31:42.603420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244380+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:31:43.244380+0000","last_peered":"2024-01-20T08:31:43.244380+0000","last_clean":"2024-01-20T08:31:43.244380+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:31:43.244380+0000","last_undegraded":"2024-01-20T08:31:43.244380+0000","last_fullsized":"2024-01-20T08:31:43.244380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985429+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:31:42.985429+0000","last_peered":"2024-01-20T08:31:42.985429+0000","last_clean":"2024-01-20T08:31:42.985429+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:31:42.985429+0000","last_undegraded":"2024-01-20T08:31:42.985429+0000","last_fullsized":"2024-01-20T08:31:42.985429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'7","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984394+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:31:42.984394+0000","last_peered":"2024-01-20T08:31:42.984394+0000","last_clean":"2024-01-20T08:31:42.984394+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:31:42.984394+0000","last_undegraded":"2024-01-20T08:31:42.984394+0000","last_fullsized":"2024-01-20T08:31:42.984394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984219+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:31:42.984219+0000","last_peered":"2024-01-20T08:31:42.984219+0000","last_clean":"2024-01-20T08:31:42.984219+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:31:42.984219+0000","last_undegraded":"2024-01-20T08:31:42.984219+0000","last_fullsized":"2024-01-20T08:31:42.984219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984827+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:31:42.984827+0000","last_peered":"2024-01-20T08:31:42.984827+0000","last_clean":"2024-01-20T08:31:42.984827+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:31:42.984827+0000","last_undegraded":"2024-01-20T08:31:42.984827+0000","last_fullsized":"2024-01-20T08:31:42.984827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.601575+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:31:42.601575+0000","last_peered":"2024-01-20T08:31:42.601575+0000","last_clean":"2024-01-20T08:31:42.601575+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:31:42.601575+0000","last_undegraded":"2024-01-20T08:31:42.601575+0000","last_fullsized":"2024-01-20T08:31:42.601575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984092+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:31:42.984092+0000","last_peered":"2024-01-20T08:31:42.984092+0000","last_clean":"2024-01-20T08:31:42.984092+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:31:42.984092+0000","last_undegraded":"2024-01-20T08:31:42.984092+0000","last_fullsized":"2024-01-20T08:31:42.984092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244558+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:31:43.244558+0000","last_peered":"2024-01-20T08:31:43.244558+0000","last_clean":"2024-01-20T08:31:43.244558+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:31:43.244558+0000","last_undegraded":"2024-01-20T08:31:43.244558+0000","last_fullsized":"2024-01-20T08:31:43.244558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"25'1","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244857+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:31:43.244857+0000","last_peered":"2024-01-20T08:31:43.244857+0000","last_clean":"2024-01-20T08:31:43.244857+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:31:43.244857+0000","last_undegraded":"2024-01-20T08:31:43.244857+0000","last_fullsized":"2024-01-20T08:31:43.244857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984898+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:31:42.984898+0000","last_peered":"2024-01-20T08:31:42.984898+0000","last_clean":"2024-01-20T08:31:42.984898+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:31:42.984898+0000","last_undegraded":"2024-01-20T08:31:42.984898+0000","last_fullsized":"2024-01-20T08:31:42.984898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978566+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:31:42.978566+0000","last_peered":"2024-01-20T08:31:42.978566+0000","last_clean":"2024-01-20T08:31:42.978566+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:31:42.978566+0000","last_undegraded":"2024-01-20T08:31:42.978566+0000","last_fullsized":"2024-01-20T08:31:42.978566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:40.141122+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:31:40.141122+0000","last_peered":"2024-01-20T08:31:40.141122+0000","last_clean":"2024-01-20T08:31:40.141122+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:31:40.141122+0000","last_undegraded":"2024-01-20T08:31:40.141122+0000","last_fullsized":"2024-01-20T08:31:40.141122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984691+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:31:42.984691+0000","last_peered":"2024-01-20T08:31:42.984691+0000","last_clean":"2024-01-20T08:31:42.984691+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:31:42.984691+0000","last_undegraded":"2024-01-20T08:31:42.984691+0000","last_fullsized":"2024-01-20T08:31:42.984691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244638+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:31:43.244638+0000","last_peered":"2024-01-20T08:31:43.244638+0000","last_clean":"2024-01-20T08:31:43.244638+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:31:43.244638+0000","last_undegraded":"2024-01-20T08:31:43.244638+0000","last_fullsized":"2024-01-20T08:31:43.244638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981553+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:31:42.981553+0000","last_peered":"2024-01-20T08:31:42.981553+0000","last_clean":"2024-01-20T08:31:42.981553+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:31:42.981553+0000","last_undegraded":"2024-01-20T08:31:42.981553+0000","last_fullsized":"2024-01-20T08:31:42.981553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984662+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:31:42.984662+0000","last_peered":"2024-01-20T08:31:42.984662+0000","last_clean":"2024-01-20T08:31:42.984662+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:31:42.984662+0000","last_undegraded":"2024-01-20T08:31:42.984662+0000","last_fullsized":"2024-01-20T08:31:42.984662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984191+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:31:42.984191+0000","last_peered":"2024-01-20T08:31:42.984191+0000","last_clean":"2024-01-20T08:31:42.984191+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:31:42.984191+0000","last_undegraded":"2024-01-20T08:31:42.984191+0000","last_fullsized":"2024-01-20T08:31:42.984191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:40.141399+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:31:40.141399+0000","last_peered":"2024-01-20T08:31:40.141399+0000","last_clean":"2024-01-20T08:31:40.141399+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:31:40.141399+0000","last_undegraded":"2024-01-20T08:31:40.141399+0000","last_fullsized":"2024-01-20T08:31:40.141399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244858+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:31:43.244858+0000","last_peered":"2024-01-20T08:31:43.244858+0000","last_clean":"2024-01-20T08:31:43.244858+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:31:43.244858+0000","last_undegraded":"2024-01-20T08:31:43.244858+0000","last_fullsized":"2024-01-20T08:31:43.244858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984705+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:31:42.984705+0000","last_peered":"2024-01-20T08:31:42.984705+0000","last_clean":"2024-01-20T08:31:42.984705+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:31:42.984705+0000","last_undegraded":"2024-01-20T08:31:42.984705+0000","last_fullsized":"2024-01-20T08:31:42.984705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'5","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603305+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:31:42.603305+0000","last_peered":"2024-01-20T08:31:42.603305+0000","last_clean":"2024-01-20T08:31:42.603305+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:31:42.603305+0000","last_undegraded":"2024-01-20T08:31:42.603305+0000","last_fullsized":"2024-01-20T08:31:42.603305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983030+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:31:42.983030+0000","last_peered":"2024-01-20T08:31:42.983030+0000","last_clean":"2024-01-20T08:31:42.983030+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:31:42.983030+0000","last_undegraded":"2024-01-20T08:31:42.983030+0000","last_fullsized":"2024-01-20T08:31:42.983030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571559+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:31:19.571559+0000","last_peered":"2024-01-20T08:31:19.571559+0000","last_clean":"2024-01-20T08:31:19.571559+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:31:19.571559+0000","last_undegraded":"2024-01-20T08:31:19.571559+0000","last_fullsized":"2024-01-20T08:31:19.571559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137086+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:31:17.137086+0000","last_peered":"2024-01-20T08:31:17.137086+0000","last_clean":"2024-01-20T08:31:17.137086+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:31:17.137086+0000","last_undegraded":"2024-01-20T08:31:17.137086+0000","last_fullsized":"2024-01-20T08:31:17.137086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984958+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:31:42.984958+0000","last_peered":"2024-01-20T08:31:42.984958+0000","last_clean":"2024-01-20T08:31:42.984958+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:31:42.984958+0000","last_undegraded":"2024-01-20T08:31:42.984958+0000","last_fullsized":"2024-01-20T08:31:42.984958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244286+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:31:43.244286+0000","last_peered":"2024-01-20T08:31:43.244286+0000","last_clean":"2024-01-20T08:31:43.244286+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:31:43.244286+0000","last_undegraded":"2024-01-20T08:31:43.244286+0000","last_fullsized":"2024-01-20T08:31:43.244286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978888+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:31:42.978888+0000","last_peered":"2024-01-20T08:31:42.978888+0000","last_clean":"2024-01-20T08:31:42.978888+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:31:42.978888+0000","last_undegraded":"2024-01-20T08:31:42.978888+0000","last_fullsized":"2024-01-20T08:31:42.978888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.803827+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:31:44.803827+0000","last_peered":"2024-01-20T08:31:44.803827+0000","last_clean":"2024-01-20T08:31:44.803827+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:31:44.803827+0000","last_undegraded":"2024-01-20T08:31:44.803827+0000","last_fullsized":"2024-01-20T08:31:44.803827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244918+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:31:43.244918+0000","last_peered":"2024-01-20T08:31:43.244918+0000","last_clean":"2024-01-20T08:31:43.244918+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:31:43.244918+0000","last_undegraded":"2024-01-20T08:31:43.244918+0000","last_fullsized":"2024-01-20T08:31:43.244918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.212803+0000","last_change":"2024-01-20T08:31:44.212803+0000","last_active":"2024-01-20T08:31:44.212803+0000","last_peered":"2024-01-20T08:31:44.212803+0000","last_clean":"2024-01-20T08:31:44.212803+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:31:44.212803+0000","last_undegraded":"2024-01-20T08:31:44.212803+0000","last_fullsized":"2024-01-20T08:31:44.212803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:29.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.812478+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:31:44.812478+0000","last_peered":"2024-01-20T08:31:44.812478+0000","last_clean":"2024-01-20T08:31:44.812478+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:31:44.812478+0000","last_undegraded":"2024-01-20T08:31:44.812478+0000","last_fullsized":"2024-01-20T08:31:44.812478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571499+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:31:19.571499+0000","last_peered":"2024-01-20T08:31:19.571499+0000","last_clean":"2024-01-20T08:31:19.571499+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:31:19.571499+0000","last_undegraded":"2024-01-20T08:31:19.571499+0000","last_fullsized":"2024-01-20T08:31:19.571499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244656+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:31:43.244656+0000","last_peered":"2024-01-20T08:31:43.244656+0000","last_clean":"2024-01-20T08:31:43.244656+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:31:43.244656+0000","last_undegraded":"2024-01-20T08:31:43.244656+0000","last_fullsized":"2024-01-20T08:31:43.244656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.023239+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:31:44.023239+0000","last_peered":"2024-01-20T08:31:44.023239+0000","last_clean":"2024-01-20T08:31:44.023239+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:31:44.023239+0000","last_undegraded":"2024-01-20T08:31:44.023239+0000","last_fullsized":"2024-01-20T08:31:44.023239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983346+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:31:42.983346+0000","last_peered":"2024-01-20T08:31:42.983346+0000","last_clean":"2024-01-20T08:31:42.983346+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:31:42.983346+0000","last_undegraded":"2024-01-20T08:31:42.983346+0000","last_fullsized":"2024-01-20T08:31:42.983346+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136152+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:31:17.136152+0000","last_peered":"2024-01-20T08:31:17.136152+0000","last_clean":"2024-01-20T08:31:17.136152+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:31:17.136152+0000","last_undegraded":"2024-01-20T08:31:17.136152+0000","last_fullsized":"2024-01-20T08:31:17.136152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'17","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.809901+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:31:44.809901+0000","last_peered":"2024-01-20T08:31:44.809901+0000","last_clean":"2024-01-20T08:31:44.809901+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:31:44.809901+0000","last_undegraded":"2024-01-20T08:31:44.809901+0000","last_fullsized":"2024-01-20T08:31:44.809901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245068+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:31:43.245068+0000","last_peered":"2024-01-20T08:31:43.245068+0000","last_clean":"2024-01-20T08:31:43.245068+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:31:43.245068+0000","last_undegraded":"2024-01-20T08:31:43.245068+0000","last_fullsized":"2024-01-20T08:31:43.245068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.982843+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:31:42.982843+0000","last_peered":"2024-01-20T08:31:42.982843+0000","last_clean":"2024-01-20T08:31:42.982843+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:31:42.982843+0000","last_undegraded":"2024-01-20T08:31:42.982843+0000","last_fullsized":"2024-01-20T08:31:42.982843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978567+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:31:42.978567+0000","last_peered":"2024-01-20T08:31:42.978567+0000","last_clean":"2024-01-20T08:31:42.978567+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:31:42.978567+0000","last_undegraded":"2024-01-20T08:31:42.978567+0000","last_fullsized":"2024-01-20T08:31:42.978567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245000+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:31:43.245000+0000","last_peered":"2024-01-20T08:31:43.245000+0000","last_clean":"2024-01-20T08:31:43.245000+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:31:43.245000+0000","last_undegraded":"2024-01-20T08:31:43.245000+0000","last_fullsized":"2024-01-20T08:31:43.245000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136632+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:31:17.136632+0000","last_peered":"2024-01-20T08:31:17.136632+0000","last_clean":"2024-01-20T08:31:17.136632+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:31:17.136632+0000","last_undegraded":"2024-01-20T08:31:17.136632+0000","last_fullsized":"2024-01-20T08:31:17.136632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603222+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:31:42.603222+0000","last_peered":"2024-01-20T08:31:42.603222+0000","last_clean":"2024-01-20T08:31:42.603222+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:31:42.603222+0000","last_undegraded":"2024-01-20T08:31:42.603222+0000","last_fullsized":"2024-01-20T08:31:42.603222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984621+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:31:42.984621+0000","last_peered":"2024-01-20T08:31:42.984621+0000","last_clean":"2024-01-20T08:31:42.984621+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:31:42.984621+0000","last_undegraded":"2024-01-20T08:31:42.984621+0000","last_fullsized":"2024-01-20T08:31:42.984621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985042+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:31:42.985042+0000","last_peered":"2024-01-20T08:31:42.985042+0000","last_clean":"2024-01-20T08:31:42.985042+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:31:42.985042+0000","last_undegraded":"2024-01-20T08:31:42.985042+0000","last_fullsized":"2024-01-20T08:31:42.985042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'7","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981351+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:31:42.981351+0000","last_peered":"2024-01-20T08:31:42.981351+0000","last_clean":"2024-01-20T08:31:42.981351+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:31:42.981351+0000","last_undegraded":"2024-01-20T08:31:42.981351+0000","last_fullsized":"2024-01-20T08:31:42.981351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571548+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:31:19.571548+0000","last_peered":"2024-01-20T08:31:19.571548+0000","last_clean":"2024-01-20T08:31:19.571548+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:31:19.571548+0000","last_undegraded":"2024-01-20T08:31:19.571548+0000","last_fullsized":"2024-01-20T08:31:19.571548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978873+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:31:42.978873+0000","last_peered":"2024-01-20T08:31:42.978873+0000","last_clean":"2024-01-20T08:31:42.978873+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:31:42.978873+0000","last_undegraded":"2024-01-20T08:31:42.978873+0000","last_fullsized":"2024-01-20T08:31:42.978873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'14","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.808950+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:31:44.808950+0000","last_peered":"2024-01-20T08:31:44.808950+0000","last_clean":"2024-01-20T08:31:44.808950+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:31:44.808950+0000","last_undegraded":"2024-01-20T08:31:44.808950+0000","last_fullsized":"2024-01-20T08:31:44.808950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984558+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:31:42.984558+0000","last_peered":"2024-01-20T08:31:42.984558+0000","last_clean":"2024-01-20T08:31:42.984558+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:31:42.984558+0000","last_undegraded":"2024-01-20T08:31:42.984558+0000","last_fullsized":"2024-01-20T08:31:42.984558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136437+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:31:17.136437+0000","last_peered":"2024-01-20T08:31:17.136437+0000","last_clean":"2024-01-20T08:31:17.136437+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:31:17.136437+0000","last_undegraded":"2024-01-20T08:31:17.136437+0000","last_fullsized":"2024-01-20T08:31:17.136437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981879+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:31:42.981879+0000","last_peered":"2024-01-20T08:31:42.981879+0000","last_clean":"2024-01-20T08:31:42.981879+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:31:42.981879+0000","last_undegraded":"2024-01-20T08:31:42.981879+0000","last_fullsized":"2024-01-20T08:31:42.981879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244323+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:31:43.244323+0000","last_peered":"2024-01-20T08:31:43.244323+0000","last_clean":"2024-01-20T08:31:43.244323+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:31:43.244323+0000","last_undegraded":"2024-01-20T08:31:43.244323+0000","last_fullsized":"2024-01-20T08:31:43.244323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245075+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:31:43.245075+0000","last_peered":"2024-01-20T08:31:43.245075+0000","last_clean":"2024-01-20T08:31:43.245075+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:31:43.245075+0000","last_undegraded":"2024-01-20T08:31:43.245075+0000","last_fullsized":"2024-01-20T08:31:43.245075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.865858+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:31:44.865858+0000","last_peered":"2024-01-20T08:31:44.865858+0000","last_clean":"2024-01-20T08:31:44.865858+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:31:44.865858+0000","last_undegraded":"2024-01-20T08:31:44.865858+0000","last_fullsized":"2024-01-20T08:31:44.865858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.980966+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:31:42.980966+0000","last_peered":"2024-01-20T08:31:42.980966+0000","last_clean":"2024-01-20T08:31:42.980966+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:31:42.980966+0000","last_undegraded":"2024-01-20T08:31:42.980966+0000","last_fullsized":"2024-01-20T08:31:42.980966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244656+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:31:43.244656+0000","last_peered":"2024-01-20T08:31:43.244656+0000","last_clean":"2024-01-20T08:31:43.244656+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:31:43.244656+0000","last_undegraded":"2024-01-20T08:31:43.244656+0000","last_fullsized":"2024-01-20T08:31:43.244656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'4","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983703+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:31:42.983703+0000","last_peered":"2024-01-20T08:31:42.983703+0000","last_clean":"2024-01-20T08:31:42.983703+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:31:42.983703+0000","last_undegraded":"2024-01-20T08:31:42.983703+0000","last_fullsized":"2024-01-20T08:31:42.983703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985885+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:31:42.985885+0000","last_peered":"2024-01-20T08:31:42.985885+0000","last_clean":"2024-01-20T08:31:42.985885+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:31:42.985885+0000","last_undegraded":"2024-01-20T08:31:42.985885+0000","last_fullsized":"2024-01-20T08:31:42.985885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984122+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:31:42.984122+0000","last_peered":"2024-01-20T08:31:42.984122+0000","last_clean":"2024-01-20T08:31:42.984122+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:31:42.984122+0000","last_undegraded":"2024-01-20T08:31:42.984122+0000","last_fullsized":"2024-01-20T08:31:42.984122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985901+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:31:42.985901+0000","last_peered":"2024-01-20T08:31:42.985901+0000","last_clean":"2024-01-20T08:31:42.985901+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:31:42.985901+0000","last_undegraded":"2024-01-20T08:31:42.985901+0000","last_fullsized":"2024-01-20T08:31:42.985901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603336+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:31:42.603336+0000","last_peered":"2024-01-20T08:31:42.603336+0000","last_clean":"2024-01-20T08:31:42.603336+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:31:42.603336+0000","last_undegraded":"2024-01-20T08:31:42.603336+0000","last_fullsized":"2024-01-20T08:31:42.603336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571569+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:31:19.571569+0000","last_peered":"2024-01-20T08:31:19.571569+0000","last_clean":"2024-01-20T08:31:19.571569+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:31:19.571569+0000","last_undegraded":"2024-01-20T08:31:19.571569+0000","last_fullsized":"2024-01-20T08:31:19.571569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244726+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:31:43.244726+0000","last_peered":"2024-01-20T08:31:43.244726+0000","last_clean":"2024-01-20T08:31:43.244726+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:31:43.244726+0000","last_undegraded":"2024-01-20T08:31:43.244726+0000","last_fullsized":"2024-01-20T08:31:43.244726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.208858+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:31:44.208858+0000","last_peered":"2024-01-20T08:31:44.208858+0000","last_clean":"2024-01-20T08:31:44.208858+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:31:44.208858+0000","last_undegraded":"2024-01-20T08:31:44.208858+0000","last_fullsized":"2024-01-20T08:31:44.208858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'7","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.605951+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:31:42.605951+0000","last_peered":"2024-01-20T08:31:42.605951+0000","last_clean":"2024-01-20T08:31:42.605951+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:31:42.605951+0000","last_undegraded":"2024-01-20T08:31:42.605951+0000","last_fullsized":"2024-01-20T08:31:42.605951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984935+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:31:42.984935+0000","last_peered":"2024-01-20T08:31:42.984935+0000","last_clean":"2024-01-20T08:31:42.984935+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:31:42.984935+0000","last_undegraded":"2024-01-20T08:31:42.984935+0000","last_fullsized":"2024-01-20T08:31:42.984935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985339+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:31:42.985339+0000","last_peered":"2024-01-20T08:31:42.985339+0000","last_clean":"2024-01-20T08:31:42.985339+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:31:42.985339+0000","last_undegraded":"2024-01-20T08:31:42.985339+0000","last_fullsized":"2024-01-20T08:31:42.985339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245102+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:31:43.245102+0000","last_peered":"2024-01-20T08:31:43.245102+0000","last_clean":"2024-01-20T08:31:43.245102+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:31:43.245102+0000","last_undegraded":"2024-01-20T08:31:43.245102+0000","last_fullsized":"2024-01-20T08:31:43.245102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983553+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:31:42.983553+0000","last_peered":"2024-01-20T08:31:42.983553+0000","last_clean":"2024-01-20T08:31:42.983553+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:31:42.983553+0000","last_undegraded":"2024-01-20T08:31:42.983553+0000","last_fullsized":"2024-01-20T08:31:42.983553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981680+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:31:42.981680+0000","last_peered":"2024-01-20T08:31:42.981680+0000","last_clean":"2024-01-20T08:31:42.981680+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:31:42.981680+0000","last_undegraded":"2024-01-20T08:31:42.981680+0000","last_fullsized":"2024-01-20T08:31:42.981680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244377+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:31:43.244377+0000","last_peered":"2024-01-20T08:31:43.244377+0000","last_clean":"2024-01-20T08:31:43.244377+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:31:43.244377+0000","last_undegraded":"2024-01-20T08:31:43.244377+0000","last_fullsized":"2024-01-20T08:31:43.244377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981454+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:31:42.981454+0000","last_peered":"2024-01-20T08:31:42.981454+0000","last_clean":"2024-01-20T08:31:42.981454+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:31:42.981454+0000","last_undegraded":"2024-01-20T08:31:42.981454+0000","last_fullsized":"2024-01-20T08:31:42.981454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136499+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:31:17.136499+0000","last_peered":"2024-01-20T08:31:17.136499+0000","last_clean":"2024-01-20T08:31:17.136499+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:31:17.136499+0000","last_undegraded":"2024-01-20T08:31:17.136499+0000","last_fullsized":"2024-01-20T08:31:17.136499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245061+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:31:43.245061+0000","last_peered":"2024-01-20T08:31:43.245061+0000","last_clean":"2024-01-20T08:31:43.245061+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:31:43.245061+0000","last_undegraded":"2024-01-20T08:31:43.245061+0000","last_fullsized":"2024-01-20T08:31:43.245061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984759+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:31:42.984759+0000","last_peered":"2024-01-20T08:31:42.984759+0000","last_clean":"2024-01-20T08:31:42.984759+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:31:42.984759+0000","last_undegraded":"2024-01-20T08:31:42.984759+0000","last_fullsized":"2024-01-20T08:31:42.984759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984413+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:31:42.984413+0000","last_peered":"2024-01-20T08:31:42.984413+0000","last_clean":"2024-01-20T08:31:42.984413+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:31:42.984413+0000","last_undegraded":"2024-01-20T08:31:42.984413+0000","last_fullsized":"2024-01-20T08:31:42.984413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244577+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:31:43.244577+0000","last_peered":"2024-01-20T08:31:43.244577+0000","last_clean":"2024-01-20T08:31:43.244577+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:31:43.244577+0000","last_undegraded":"2024-01-20T08:31:43.244577+0000","last_fullsized":"2024-01-20T08:31:43.244577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244973+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:31:43.244973+0000","last_peered":"2024-01-20T08:31:43.244973+0000","last_clean":"2024-01-20T08:31:43.244973+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:31:43.244973+0000","last_undegraded":"2024-01-20T08:31:43.244973+0000","last_fullsized":"2024-01-20T08:31:43.244973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136221+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:31:17.136221+0000","last_peered":"2024-01-20T08:31:17.136221+0000","last_clean":"2024-01-20T08:31:17.136221+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:31:17.136221+0000","last_undegraded":"2024-01-20T08:31:17.136221+0000","last_fullsized":"2024-01-20T08:31:17.136221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244645+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:31:43.244645+0000","last_peered":"2024-01-20T08:31:43.244645+0000","last_clean":"2024-01-20T08:31:43.244645+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:31:43.244645+0000","last_undegraded":"2024-01-20T08:31:43.244645+0000","last_fullsized":"2024-01-20T08:31:43.244645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983059+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:31:42.983059+0000","last_peered":"2024-01-20T08:31:42.983059+0000","last_clean":"2024-01-20T08:31:42.983059+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:31:42.983059+0000","last_undegraded":"2024-01-20T08:31:42.983059+0000","last_fullsized":"2024-01-20T08:31:42.983059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984863+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:31:42.984863+0000","last_peered":"2024-01-20T08:31:42.984863+0000","last_clean":"2024-01-20T08:31:42.984863+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:31:42.984863+0000","last_undegraded":"2024-01-20T08:31:42.984863+0000","last_fullsized":"2024-01-20T08:31:42.984863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978831+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:31:42.978831+0000","last_peered":"2024-01-20T08:31:42.978831+0000","last_clean":"2024-01-20T08:31:42.978831+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:31:42.978831+0000","last_undegraded":"2024-01-20T08:31:42.978831+0000","last_fullsized":"2024-01-20T08:31:42.978831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137287+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:31:17.137287+0000","last_peered":"2024-01-20T08:31:17.137287+0000","last_clean":"2024-01-20T08:31:17.137287+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:31:17.137287+0000","last_undegraded":"2024-01-20T08:31:17.137287+0000","last_fullsized":"2024-01-20T08:31:17.137287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981474+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:31:42.981474+0000","last_peered":"2024-01-20T08:31:42.981474+0000","last_clean":"2024-01-20T08:31:42.981474+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:31:42.981474+0000","last_undegraded":"2024-01-20T08:31:42.981474+0000","last_fullsized":"2024-01-20T08:31:42.981474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136720+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:31:17.136720+0000","last_peered":"2024-01-20T08:31:17.136720+0000","last_clean":"2024-01-20T08:31:17.136720+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:31:17.136720+0000","last_undegraded":"2024-01-20T08:31:17.136720+0000","last_fullsized":"2024-01-20T08:31:17.136720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136207+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:31:17.136207+0000","last_peered":"2024-01-20T08:31:17.136207+0000","last_clean":"2024-01-20T08:31:17.136207+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:31:17.136207+0000","last_undegraded":"2024-01-20T08:31:17.136207+0000","last_fullsized":"2024-01-20T08:31:17.136207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244598+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:31:43.244598+0000","last_peered":"2024-01-20T08:31:43.244598+0000","last_clean":"2024-01-20T08:31:43.244598+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:31:43.244598+0000","last_undegraded":"2024-01-20T08:31:43.244598+0000","last_fullsized":"2024-01-20T08:31:43.244598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984971+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:31:42.984971+0000","last_peered":"2024-01-20T08:31:42.984971+0000","last_clean":"2024-01-20T08:31:42.984971+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:31:42.984971+0000","last_undegraded":"2024-01-20T08:31:42.984971+0000","last_fullsized":"2024-01-20T08:31:42.984971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984181+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:31:42.984181+0000","last_peered":"2024-01-20T08:31:42.984181+0000","last_clean":"2024-01-20T08:31:42.984181+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:31:42.984181+0000","last_undegraded":"2024-01-20T08:31:42.984181+0000","last_fullsized":"2024-01-20T08:31:42.984181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978794+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:31:42.978794+0000","last_peered":"2024-01-20T08:31:42.978794+0000","last_clean":"2024-01-20T08:31:42.978794+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:31:42.978794+0000","last_undegraded":"2024-01-20T08:31:42.978794+0000","last_fullsized":"2024-01-20T08:31:42.978794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.979087+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:31:42.979087+0000","last_peered":"2024-01-20T08:31:42.979087+0000","last_clean":"2024-01-20T08:31:42.979087+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:31:42.979087+0000","last_undegraded":"2024-01-20T08:31:42.979087+0000","last_fullsized":"2024-01-20T08:31:42.979087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978943+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:31:42.978943+0000","last_peered":"2024-01-20T08:31:42.978943+0000","last_clean":"2024-01-20T08:31:42.978943+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:31:42.978943+0000","last_undegraded":"2024-01-20T08:31:42.978943+0000","last_fullsized":"2024-01-20T08:31:42.978943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983499+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:31:42.983499+0000","last_peered":"2024-01-20T08:31:42.983499+0000","last_clean":"2024-01-20T08:31:42.983499+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:31:42.983499+0000","last_undegraded":"2024-01-20T08:31:42.983499+0000","last_fullsized":"2024-01-20T08:31:42.983499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244621+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:31:43.244621+0000","last_peered":"2024-01-20T08:31:43.244621+0000","last_clean":"2024-01-20T08:31:43.244621+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:31:43.244621+0000","last_undegraded":"2024-01-20T08:31:43.244621+0000","last_fullsized":"2024-01-20T08:31:43.244621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978481+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:31:42.978481+0000","last_peered":"2024-01-20T08:31:42.978481+0000","last_clean":"2024-01-20T08:31:42.978481+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:31:42.978481+0000","last_undegraded":"2024-01-20T08:31:42.978481+0000","last_fullsized":"2024-01-20T08:31:42.978481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244425+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:31:43.244425+0000","last_peered":"2024-01-20T08:31:43.244425+0000","last_clean":"2024-01-20T08:31:43.244425+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:31:43.244425+0000","last_undegraded":"2024-01-20T08:31:43.244425+0000","last_fullsized":"2024-01-20T08:31:43.244425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978643+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:31:42.978643+0000","last_peered":"2024-01-20T08:31:42.978643+0000","last_clean":"2024-01-20T08:31:42.978643+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:31:42.978643+0000","last_undegraded":"2024-01-20T08:31:42.978643+0000","last_fullsized":"2024-01-20T08:31:42.978643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244404+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:31:43.244404+0000","last_peered":"2024-01-20T08:31:43.244404+0000","last_clean":"2024-01-20T08:31:43.244404+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:31:43.244404+0000","last_undegraded":"2024-01-20T08:31:43.244404+0000","last_fullsized":"2024-01-20T08:31:43.244404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244863+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:31:43.244863+0000","last_peered":"2024-01-20T08:31:43.244863+0000","last_clean":"2024-01-20T08:31:43.244863+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:31:43.244863+0000","last_undegraded":"2024-01-20T08:31:43.244863+0000","last_fullsized":"2024-01-20T08:31:43.244863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244967+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:31:43.244967+0000","last_peered":"2024-01-20T08:31:43.244967+0000","last_clean":"2024-01-20T08:31:43.244967+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:31:43.244967+0000","last_undegraded":"2024-01-20T08:31:43.244967+0000","last_fullsized":"2024-01-20T08:31:43.244967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978956+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:31:42.978956+0000","last_peered":"2024-01-20T08:31:42.978956+0000","last_clean":"2024-01-20T08:31:42.978956+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:31:42.978956+0000","last_undegraded":"2024-01-20T08:31:42.978956+0000","last_fullsized":"2024-01-20T08:31:42.978956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9529,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":19702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168,"ondisk_log_size":168,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":119853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":708667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":123152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":121382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":709255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":319488,"data_stored":118505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:31:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":112659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:31:49.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:31:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 Processor -- start 2024-01-20T08:31:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 -- start start 2024-01-20T08:31:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 0x7f103406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10340c8150 0x7f103406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1034157490 con 0x7f1034151a00 2024-01-20T08:31:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103406cff0 con 0x7f10340c8150 2024-01-20T08:31:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.490+0000 7f103a7e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10341576c0 con 0x7f1034151620 2024-01-20T08:31:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.491+0000 7f1033fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 0x7f103406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.491+0000 7f1033fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 0x7f103406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47386/0 (socket says 172.21.15.87:47386) 2024-01-20T08:31:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.491+0000 7f1033fff640 1 -- 172.21.15.87:0/3800772246 learned_addr learned my addr 172.21.15.87:0/3800772246 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.491+0000 7f1038d82640 1 --2- 172.21.15.87:0/3800772246 >> v2:172.21.15.116:3300/0 conn(0x7f10340c8150 0x7f103406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.494+0000 7f1033fff640 1 -- 172.21.15.87:0/3800772246 >> v2:172.21.15.116:3300/0 conn(0x7f10340c8150 msgr2=0x7f103406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:49.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.494+0000 7f1033fff640 1 --2- 172.21.15.87:0/3800772246 >> v2:172.21.15.116:3300/0 conn(0x7f10340c8150 0x7f103406d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.494+0000 7f1033fff640 1 -- 172.21.15.87:0/3800772246 >> v1:172.21.15.116:6789/0 conn(0x7f1034151a00 legacy=0x7f10340c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.494+0000 7f1033fff640 1 -- 172.21.15.87:0/3800772246 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10341579a0 con 0x7f1034151620 2024-01-20T08:31:49.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.496+0000 7f1033fff640 1 --2- 172.21.15.87:0/3800772246 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 0x7f103406f550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1028009870 tx=0x7f102802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af41334406175e9d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f1032ffd640 1 -- 172.21.15.87:0/3800772246 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028030840 con 0x7f1034151620 2024-01-20T08:31:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f1032ffd640 1 -- 172.21.15.87:0/3800772246 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10280309a0 con 0x7f1034151620 2024-01-20T08:31:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f1032ffd640 1 -- 172.21.15.87:0/3800772246 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028030d30 con 0x7f1034151620 2024-01-20T08:31:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 -- 172.21.15.87:0/3800772246 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 msgr2=0x7f103406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3800772246 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 0x7f103406f550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1028009870 tx=0x7f102802b980 comp rx=0 tx=0).stop 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 -- 172.21.15.87:0/3800772246 shutdown_connections 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3800772246 >> v2:172.21.15.116:3300/0 conn(0x7f10340c8150 0x7f103406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3800772246 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151620 0x7f103406f550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 -- 172.21.15.87:0/3800772246 >> 172.21.15.87:0/3800772246 conn(0x7f103406abe0 msgr2=0x7f103406ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 -- 172.21.15.87:0/3800772246 shutdown_connections 2024-01-20T08:31:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.500+0000 7f103a7e5640 1 -- 172.21.15.87:0/3800772246 wait complete. 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.501+0000 7f103a7e5640 1 Processor -- start 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.501+0000 7f103a7e5640 1 -- start start 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.502+0000 7f103a7e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10340c8150 0x7f103414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.502+0000 7f103a7e5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1034151620 0x7f10340c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.502+0000 7f103a7e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 0x7f1034150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.502+0000 7f103a7e5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341f0790 con 0x7f1034151620 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.502+0000 7f103a7e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341570d0 con 0x7f10340c8150 2024-01-20T08:31:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.502+0000 7f103a7e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10341ed2f0 con 0x7f1034151a00 2024-01-20T08:31:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.503+0000 7f1038d82640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 0x7f1034150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.503+0000 7f1033fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10340c8150 0x7f103414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.503+0000 7f10337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1034151620 0x7f10340c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.503+0000 7f1038d82640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 0x7f1034150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47398/0 (socket says 172.21.15.87:47398) 2024-01-20T08:31:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.503+0000 7f1038d82640 1 -- 172.21.15.87:0/3090411377 learned_addr learned my addr 172.21.15.87:0/3090411377 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:49.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.506+0000 7f1038d82640 1 -- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10340c8150 msgr2=0x7f103414cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.506+0000 7f1038d82640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10340c8150 0x7f103414cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.506+0000 7f1038d82640 1 -- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1034151620 msgr2=0x7f10340c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.506+0000 7f1038d82640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1034151620 0x7f10340c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.506+0000 7f1038d82640 1 -- 172.21.15.87:0/3090411377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280094d0 con 0x7f1034151a00 2024-01-20T08:31:49.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.508+0000 7f1038d82640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 0x7f1034150430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f102c007e30 tx=0x7f102c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.512+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c014620 con 0x7f1034151a00 2024-01-20T08:31:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.512+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c00ee90 con 0x7f1034151a00 2024-01-20T08:31:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.512+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10341f0bb0 con 0x7f1034151a00 2024-01-20T08:31:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.512+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0149c0 con 0x7f1034151a00 2024-01-20T08:31:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.512+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341f00d0 con 0x7f1034151a00 2024-01-20T08:31:49.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.513+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1034067260 con 0x7f1034151a00 2024-01-20T08:31:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.531+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f102c014b20 con 0x7f1034151a00 2024-01-20T08:31:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.531+0000 7f1030ff9640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f102006af20 0x7f102006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.532+0000 7f1033fff640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f102006af20 0x7f102006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.532+0000 7f1033fff640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f102006af20 0x7f102006d3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1028036000 tx=0x7f1028008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.534+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f102c059520 con 0x7f1034151a00 2024-01-20T08:31:49.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.774+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f102c055860 con 0x7f1034151a00 2024-01-20T08:31:49.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.942+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f103406dfe0 con 0x7f102006af20 2024-01-20T08:31:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.950+0000 7f1030ff9640 1 -- 172.21.15.87:0/3090411377 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345827 (secure 0 0 0) 0x7f103406dfe0 con 0x7f102006af20 2024-01-20T08:31:49.957 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:49.958 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f102006af20 msgr2=0x7f102006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f102006af20 0x7f102006d3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1028036000 tx=0x7f1028008030 comp rx=0 tx=0).stop 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 msgr2=0x7f1034150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 0x7f1034150430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f102c007e30 tx=0x7f102c00e4b0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 shutdown_connections 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f102006af20 0x7f102006d3e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1034151a00 0x7f1034150430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1034151620 0x7f10340c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 --2- 172.21.15.87:0/3090411377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10340c8150 0x7f103414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:49.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.953+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 >> 172.21.15.87:0/3090411377 conn(0x7f103406abe0 msgr2=0x7f10340c4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:49.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.954+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 shutdown_connections 2024-01-20T08:31:49.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:49.954+0000 7f103a7e5640 1 -- 172.21.15.87:0/3090411377 wait complete. 2024-01-20T08:31:49.967 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-01-20T08:31:48.286014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599897,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":353,"num_write_kb":2479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":324,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219792,"kb_used_data":3624,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754928,"statfs":{"total":773094113280,"available":772869046272,"internally_reserved":0,"allocated":3710976,"data_stored":2133488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4286,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002149"},"pg_stats":[{"pgid":"2.1f","version":"27'13","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.812971+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:31:44.812971+0000","last_peered":"2024-01-20T08:31:44.812971+0000","last_clean":"2024-01-20T08:31:44.812971+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:31:44.812971+0000","last_undegraded":"2024-01-20T08:31:44.812971+0000","last_fullsized":"2024-01-20T08:31:44.812971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245064+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:31:43.245064+0000","last_peered":"2024-01-20T08:31:43.245064+0000","last_clean":"2024-01-20T08:31:43.245064+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:31:43.245064+0000","last_undegraded":"2024-01-20T08:31:43.245064+0000","last_fullsized":"2024-01-20T08:31:43.245064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.811554+0000","last_change":"2024-01-20T08:31:44.811554+0000","last_active":"2024-01-20T08:31:44.811554+0000","last_peered":"2024-01-20T08:31:44.811554+0000","last_clean":"2024-01-20T08:31:44.811554+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:31:44.811554+0000","last_undegraded":"2024-01-20T08:31:44.811554+0000","last_fullsized":"2024-01-20T08:31:44.811554+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:31:34.560783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245144+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:31:43.245144+0000","last_peered":"2024-01-20T08:31:43.245144+0000","last_clean":"2024-01-20T08:31:43.245144+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:31:43.245144+0000","last_undegraded":"2024-01-20T08:31:43.245144+0000","last_fullsized":"2024-01-20T08:31:43.245144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245051+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:31:43.245051+0000","last_peered":"2024-01-20T08:31:43.245051+0000","last_clean":"2024-01-20T08:31:43.245051+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:31:43.245051+0000","last_undegraded":"2024-01-20T08:31:43.245051+0000","last_fullsized":"2024-01-20T08:31:43.245051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244170+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:31:43.244170+0000","last_peered":"2024-01-20T08:31:43.244170+0000","last_clean":"2024-01-20T08:31:43.244170+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:31:43.244170+0000","last_undegraded":"2024-01-20T08:31:43.244170+0000","last_fullsized":"2024-01-20T08:31:43.244170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.893516+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:31:44.893516+0000","last_peered":"2024-01-20T08:31:44.893516+0000","last_clean":"2024-01-20T08:31:44.893516+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:31:44.893516+0000","last_undegraded":"2024-01-20T08:31:44.893516+0000","last_fullsized":"2024-01-20T08:31:44.893516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983392+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:31:42.983392+0000","last_peered":"2024-01-20T08:31:42.983392+0000","last_clean":"2024-01-20T08:31:42.983392+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:31:42.983392+0000","last_undegraded":"2024-01-20T08:31:42.983392+0000","last_fullsized":"2024-01-20T08:31:42.983392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244350+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:31:43.244350+0000","last_peered":"2024-01-20T08:31:43.244350+0000","last_clean":"2024-01-20T08:31:43.244350+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:31:43.244350+0000","last_undegraded":"2024-01-20T08:31:43.244350+0000","last_fullsized":"2024-01-20T08:31:43.244350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.893215+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:31:44.893215+0000","last_peered":"2024-01-20T08:31:44.893215+0000","last_clean":"2024-01-20T08:31:44.893215+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:31:44.893215+0000","last_undegraded":"2024-01-20T08:31:44.893215+0000","last_fullsized":"2024-01-20T08:31:44.893215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137012+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:31:17.137012+0000","last_peered":"2024-01-20T08:31:17.137012+0000","last_clean":"2024-01-20T08:31:17.137012+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:31:17.137012+0000","last_undegraded":"2024-01-20T08:31:17.137012+0000","last_fullsized":"2024-01-20T08:31:17.137012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244287+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:31:43.244287+0000","last_peered":"2024-01-20T08:31:43.244287+0000","last_clean":"2024-01-20T08:31:43.244287+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:31:43.244287+0000","last_undegraded":"2024-01-20T08:31:43.244287+0000","last_fullsized":"2024-01-20T08:31:43.244287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136470+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:31:17.136470+0000","last_peered":"2024-01-20T08:31:17.136470+0000","last_clean":"2024-01-20T08:31:17.136470+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:31:17.136470+0000","last_undegraded":"2024-01-20T08:31:17.136470+0000","last_fullsized":"2024-01-20T08:31:17.136470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:37.999583+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:31:37.999583+0000","last_peered":"2024-01-20T08:31:37.999583+0000","last_clean":"2024-01-20T08:31:37.999583+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:31:37.999583+0000","last_undegraded":"2024-01-20T08:31:37.999583+0000","last_fullsized":"2024-01-20T08:31:37.999583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571539+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:31:19.571539+0000","last_peered":"2024-01-20T08:31:19.571539+0000","last_clean":"2024-01-20T08:31:19.571539+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:31:19.571539+0000","last_undegraded":"2024-01-20T08:31:19.571539+0000","last_fullsized":"2024-01-20T08:31:19.571539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981209+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:31:42.981209+0000","last_peered":"2024-01-20T08:31:42.981209+0000","last_clean":"2024-01-20T08:31:42.981209+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:31:42.981209+0000","last_undegraded":"2024-01-20T08:31:42.981209+0000","last_fullsized":"2024-01-20T08:31:42.981209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.209035+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:31:44.209035+0000","last_peered":"2024-01-20T08:31:44.209035+0000","last_clean":"2024-01-20T08:31:44.209035+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:31:44.209035+0000","last_undegraded":"2024-01-20T08:31:44.209035+0000","last_fullsized":"2024-01-20T08:31:44.209035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978645+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:31:42.978645+0000","last_peered":"2024-01-20T08:31:42.978645+0000","last_clean":"2024-01-20T08:31:42.978645+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:31:42.978645+0000","last_undegraded":"2024-01-20T08:31:42.978645+0000","last_fullsized":"2024-01-20T08:31:42.978645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984304+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:31:42.984304+0000","last_peered":"2024-01-20T08:31:42.984304+0000","last_clean":"2024-01-20T08:31:42.984304+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:31:42.984304+0000","last_undegraded":"2024-01-20T08:31:42.984304+0000","last_fullsized":"2024-01-20T08:31:42.984304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.812638+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:31:44.812638+0000","last_peered":"2024-01-20T08:31:44.812638+0000","last_clean":"2024-01-20T08:31:44.812638+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:31:44.812638+0000","last_undegraded":"2024-01-20T08:31:44.812638+0000","last_fullsized":"2024-01-20T08:31:44.812638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981846+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:31:42.981846+0000","last_peered":"2024-01-20T08:31:42.981846+0000","last_clean":"2024-01-20T08:31:42.981846+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:31:42.981846+0000","last_undegraded":"2024-01-20T08:31:42.981846+0000","last_fullsized":"2024-01-20T08:31:42.981846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136221+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:31:17.136221+0000","last_peered":"2024-01-20T08:31:17.136221+0000","last_clean":"2024-01-20T08:31:17.136221+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:31:17.136221+0000","last_undegraded":"2024-01-20T08:31:17.136221+0000","last_fullsized":"2024-01-20T08:31:17.136221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978331+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:31:42.978331+0000","last_peered":"2024-01-20T08:31:42.978331+0000","last_clean":"2024-01-20T08:31:42.978331+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:31:42.978331+0000","last_undegraded":"2024-01-20T08:31:42.978331+0000","last_fullsized":"2024-01-20T08:31:42.978331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244578+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:31:43.244578+0000","last_peered":"2024-01-20T08:31:43.244578+0000","last_clean":"2024-01-20T08:31:43.244578+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:31:43.244578+0000","last_undegraded":"2024-01-20T08:31:43.244578+0000","last_fullsized":"2024-01-20T08:31:43.244578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984212+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:31:42.984212+0000","last_peered":"2024-01-20T08:31:42.984212+0000","last_clean":"2024-01-20T08:31:42.984212+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:31:42.984212+0000","last_undegraded":"2024-01-20T08:31:42.984212+0000","last_fullsized":"2024-01-20T08:31:42.984212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'5","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603420+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:31:42.603420+0000","last_peered":"2024-01-20T08:31:42.603420+0000","last_clean":"2024-01-20T08:31:42.603420+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:31:42.603420+0000","last_undegraded":"2024-01-20T08:31:42.603420+0000","last_fullsized":"2024-01-20T08:31:42.603420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244380+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:31:43.244380+0000","last_peered":"2024-01-20T08:31:43.244380+0000","last_clean":"2024-01-20T08:31:43.244380+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:31:43.244380+0000","last_undegraded":"2024-01-20T08:31:43.244380+0000","last_fullsized":"2024-01-20T08:31:43.244380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985429+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:31:42.985429+0000","last_peered":"2024-01-20T08:31:42.985429+0000","last_clean":"2024-01-20T08:31:42.985429+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:31:42.985429+0000","last_undegraded":"2024-01-20T08:31:42.985429+0000","last_fullsized":"2024-01-20T08:31:42.985429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'7","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984394+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:31:42.984394+0000","last_peered":"2024-01-20T08:31:42.984394+0000","last_clean":"2024-01-20T08:31:42.984394+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:31:42.984394+0000","last_undegraded":"2024-01-20T08:31:42.984394+0000","last_fullsized":"2024-01-20T08:31:42.984394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984219+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:31:42.984219+0000","last_peered":"2024-01-20T08:31:42.984219+0000","last_clean":"2024-01-20T08:31:42.984219+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:31:42.984219+0000","last_undegraded":"2024-01-20T08:31:42.984219+0000","last_fullsized":"2024-01-20T08:31:42.984219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984827+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:31:42.984827+0000","last_peered":"2024-01-20T08:31:42.984827+0000","last_clean":"2024-01-20T08:31:42.984827+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:31:42.984827+0000","last_undegraded":"2024-01-20T08:31:42.984827+0000","last_fullsized":"2024-01-20T08:31:42.984827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.601575+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:31:42.601575+0000","last_peered":"2024-01-20T08:31:42.601575+0000","last_clean":"2024-01-20T08:31:42.601575+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:31:42.601575+0000","last_undegraded":"2024-01-20T08:31:42.601575+0000","last_fullsized":"2024-01-20T08:31:42.601575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984092+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:31:42.984092+0000","last_peered":"2024-01-20T08:31:42.984092+0000","last_clean":"2024-01-20T08:31:42.984092+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:31:42.984092+0000","last_undegraded":"2024-01-20T08:31:42.984092+0000","last_fullsized":"2024-01-20T08:31:42.984092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244558+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:31:43.244558+0000","last_peered":"2024-01-20T08:31:43.244558+0000","last_clean":"2024-01-20T08:31:43.244558+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:31:43.244558+0000","last_undegraded":"2024-01-20T08:31:43.244558+0000","last_fullsized":"2024-01-20T08:31:43.244558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"25'1","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244857+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:31:43.244857+0000","last_peered":"2024-01-20T08:31:43.244857+0000","last_clean":"2024-01-20T08:31:43.244857+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:31:43.244857+0000","last_undegraded":"2024-01-20T08:31:43.244857+0000","last_fullsized":"2024-01-20T08:31:43.244857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984898+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:31:42.984898+0000","last_peered":"2024-01-20T08:31:42.984898+0000","last_clean":"2024-01-20T08:31:42.984898+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:31:42.984898+0000","last_undegraded":"2024-01-20T08:31:42.984898+0000","last_fullsized":"2024-01-20T08:31:42.984898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978566+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:31:42.978566+0000","last_peered":"2024-01-20T08:31:42.978566+0000","last_clean":"2024-01-20T08:31:42.978566+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:31:42.978566+0000","last_undegraded":"2024-01-20T08:31:42.978566+0000","last_fullsized":"2024-01-20T08:31:42.978566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:40.141122+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:31:40.141122+0000","last_peered":"2024-01-20T08:31:40.141122+0000","last_clean":"2024-01-20T08:31:40.141122+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:31:40.141122+0000","last_undegraded":"2024-01-20T08:31:40.141122+0000","last_fullsized":"2024-01-20T08:31:40.141122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984691+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:31:42.984691+0000","last_peered":"2024-01-20T08:31:42.984691+0000","last_clean":"2024-01-20T08:31:42.984691+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:31:42.984691+0000","last_undegraded":"2024-01-20T08:31:42.984691+0000","last_fullsized":"2024-01-20T08:31:42.984691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244638+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:31:43.244638+0000","last_peered":"2024-01-20T08:31:43.244638+0000","last_clean":"2024-01-20T08:31:43.244638+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:31:43.244638+0000","last_undegraded":"2024-01-20T08:31:43.244638+0000","last_fullsized":"2024-01-20T08:31:43.244638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981553+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:31:42.981553+0000","last_peered":"2024-01-20T08:31:42.981553+0000","last_clean":"2024-01-20T08:31:42.981553+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:31:42.981553+0000","last_undegraded":"2024-01-20T08:31:42.981553+0000","last_fullsized":"2024-01-20T08:31:42.981553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984662+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:31:42.984662+0000","last_peered":"2024-01-20T08:31:42.984662+0000","last_clean":"2024-01-20T08:31:42.984662+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:31:42.984662+0000","last_undegraded":"2024-01-20T08:31:42.984662+0000","last_fullsized":"2024-01-20T08:31:42.984662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984191+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:31:42.984191+0000","last_peered":"2024-01-20T08:31:42.984191+0000","last_clean":"2024-01-20T08:31:42.984191+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:31:42.984191+0000","last_undegraded":"2024-01-20T08:31:42.984191+0000","last_fullsized":"2024-01-20T08:31:42.984191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:40.141399+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:31:40.141399+0000","last_peered":"2024-01-20T08:31:40.141399+0000","last_clean":"2024-01-20T08:31:40.141399+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:31:40.141399+0000","last_undegraded":"2024-01-20T08:31:40.141399+0000","last_fullsized":"2024-01-20T08:31:40.141399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244858+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:31:43.244858+0000","last_peered":"2024-01-20T08:31:43.244858+0000","last_clean":"2024-01-20T08:31:43.244858+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:31:43.244858+0000","last_undegraded":"2024-01-20T08:31:43.244858+0000","last_fullsized":"2024-01-20T08:31:43.244858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984705+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:31:42.984705+0000","last_peered":"2024-01-20T08:31:42.984705+0000","last_clean":"2024-01-20T08:31:42.984705+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:31:42.984705+0000","last_undegraded":"2024-01-20T08:31:42.984705+0000","last_fullsized":"2024-01-20T08:31:42.984705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'5","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603305+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:31:42.603305+0000","last_peered":"2024-01-20T08:31:42.603305+0000","last_clean":"2024-01-20T08:31:42.603305+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:31:42.603305+0000","last_undegraded":"2024-01-20T08:31:42.603305+0000","last_fullsized":"2024-01-20T08:31:42.603305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983030+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:31:42.983030+0000","last_peered":"2024-01-20T08:31:42.983030+0000","last_clean":"2024-01-20T08:31:42.983030+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:31:42.983030+0000","last_undegraded":"2024-01-20T08:31:42.983030+0000","last_fullsized":"2024-01-20T08:31:42.983030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571559+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:31:19.571559+0000","last_peered":"2024-01-20T08:31:19.571559+0000","last_clean":"2024-01-20T08:31:19.571559+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:31:19.571559+0000","last_undegraded":"2024-01-20T08:31:19.571559+0000","last_fullsized":"2024-01-20T08:31:19.571559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137086+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:31:17.137086+0000","last_peered":"2024-01-20T08:31:17.137086+0000","last_clean":"2024-01-20T08:31:17.137086+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:31:17.137086+0000","last_undegraded":"2024-01-20T08:31:17.137086+0000","last_fullsized":"2024-01-20T08:31:17.137086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984958+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:31:42.984958+0000","last_peered":"2024-01-20T08:31:42.984958+0000","last_clean":"2024-01-20T08:31:42.984958+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:31:42.984958+0000","last_undegraded":"2024-01-20T08:31:42.984958+0000","last_fullsized":"2024-01-20T08:31:42.984958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244286+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:31:43.244286+0000","last_peered":"2024-01-20T08:31:43.244286+0000","last_clean":"2024-01-20T08:31:43.244286+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:31:43.244286+0000","last_undegraded":"2024-01-20T08:31:43.244286+0000","last_fullsized":"2024-01-20T08:31:43.244286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978888+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:31:42.978888+0000","last_peered":"2024-01-20T08:31:42.978888+0000","last_clean":"2024-01-20T08:31:42.978888+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:31:42.978888+0000","last_undegraded":"2024-01-20T08:31:42.978888+0000","last_fullsized":"2024-01-20T08:31:42.978888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.803827+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:31:44.803827+0000","last_peered":"2024-01-20T08:31:44.803827+0000","last_clean":"2024-01-20T08:31:44.803827+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:31:44.803827+0000","last_undegraded":"2024-01-20T08:31:44.803827+0000","last_fullsized":"2024-01-20T08:31:44.803827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244918+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:31:43.244918+0000","last_peered":"2024-01-20T08:31:43.244918+0000","last_clean":"2024-01-20T08:31:43.244918+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:31:43.244918+0000","last_undegraded":"2024-01-20T08:31:43.244918+0000","last_fullsized":"2024-01-20T08:31:43.244918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.212803+0000","last_change":"2024-01-20T08:31:44.212803+0000","last_active":"2024-01-20T08:31:44.212803+0000","last_peered":"2024-01-20T08:31:44.212803+0000","last_clean":"2024-01-20T08:31:44.212803+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:31:44.212803+0000","last_undegraded":"2024-01-20T08:31:44.212803+0000","last_fullsized":"2024-01-20T08:31:44.212803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:29.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.812478+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:31:44.812478+0000","last_peered":"2024-01-20T08:31:44.812478+0000","last_clean":"2024-01-20T08:31:44.812478+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:31:44.812478+0000","last_undegraded":"2024-01-20T08:31:44.812478+0000","last_fullsized":"2024-01-20T08:31:44.812478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571499+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:31:19.571499+0000","last_peered":"2024-01-20T08:31:19.571499+0000","last_clean":"2024-01-20T08:31:19.571499+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:31:19.571499+0000","last_undegraded":"2024-01-20T08:31:19.571499+0000","last_fullsized":"2024-01-20T08:31:19.571499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244656+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:31:43.244656+0000","last_peered":"2024-01-20T08:31:43.244656+0000","last_clean":"2024-01-20T08:31:43.244656+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:31:43.244656+0000","last_undegraded":"2024-01-20T08:31:43.244656+0000","last_fullsized":"2024-01-20T08:31:43.244656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.023239+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:31:44.023239+0000","last_peered":"2024-01-20T08:31:44.023239+0000","last_clean":"2024-01-20T08:31:44.023239+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:31:44.023239+0000","last_undegraded":"2024-01-20T08:31:44.023239+0000","last_fullsized":"2024-01-20T08:31:44.023239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983346+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:31:42.983346+0000","last_peered":"2024-01-20T08:31:42.983346+0000","last_clean":"2024-01-20T08:31:42.983346+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:31:42.983346+0000","last_undegraded":"2024-01-20T08:31:42.983346+0000","last_fullsized":"2024-01-20T08:31:42.983346+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136152+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:31:17.136152+0000","last_peered":"2024-01-20T08:31:17.136152+0000","last_clean":"2024-01-20T08:31:17.136152+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:31:17.136152+0000","last_undegraded":"2024-01-20T08:31:17.136152+0000","last_fullsized":"2024-01-20T08:31:17.136152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'17","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.809901+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:31:44.809901+0000","last_peered":"2024-01-20T08:31:44.809901+0000","last_clean":"2024-01-20T08:31:44.809901+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:31:44.809901+0000","last_undegraded":"2024-01-20T08:31:44.809901+0000","last_fullsized":"2024-01-20T08:31:44.809901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245068+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:31:43.245068+0000","last_peered":"2024-01-20T08:31:43.245068+0000","last_clean":"2024-01-20T08:31:43.245068+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:31:43.245068+0000","last_undegraded":"2024-01-20T08:31:43.245068+0000","last_fullsized":"2024-01-20T08:31:43.245068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.982843+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:31:42.982843+0000","last_peered":"2024-01-20T08:31:42.982843+0000","last_clean":"2024-01-20T08:31:42.982843+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:31:42.982843+0000","last_undegraded":"2024-01-20T08:31:42.982843+0000","last_fullsized":"2024-01-20T08:31:42.982843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978567+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:31:42.978567+0000","last_peered":"2024-01-20T08:31:42.978567+0000","last_clean":"2024-01-20T08:31:42.978567+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:31:42.978567+0000","last_undegraded":"2024-01-20T08:31:42.978567+0000","last_fullsized":"2024-01-20T08:31:42.978567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245000+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:31:43.245000+0000","last_peered":"2024-01-20T08:31:43.245000+0000","last_clean":"2024-01-20T08:31:43.245000+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:31:43.245000+0000","last_undegraded":"2024-01-20T08:31:43.245000+0000","last_fullsized":"2024-01-20T08:31:43.245000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136632+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:31:17.136632+0000","last_peered":"2024-01-20T08:31:17.136632+0000","last_clean":"2024-01-20T08:31:17.136632+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:31:17.136632+0000","last_undegraded":"2024-01-20T08:31:17.136632+0000","last_fullsized":"2024-01-20T08:31:17.136632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603222+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:31:42.603222+0000","last_peered":"2024-01-20T08:31:42.603222+0000","last_clean":"2024-01-20T08:31:42.603222+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:31:42.603222+0000","last_undegraded":"2024-01-20T08:31:42.603222+0000","last_fullsized":"2024-01-20T08:31:42.603222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984621+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:31:42.984621+0000","last_peered":"2024-01-20T08:31:42.984621+0000","last_clean":"2024-01-20T08:31:42.984621+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:31:42.984621+0000","last_undegraded":"2024-01-20T08:31:42.984621+0000","last_fullsized":"2024-01-20T08:31:42.984621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985042+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:31:42.985042+0000","last_peered":"2024-01-20T08:31:42.985042+0000","last_clean":"2024-01-20T08:31:42.985042+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:31:42.985042+0000","last_undegraded":"2024-01-20T08:31:42.985042+0000","last_fullsized":"2024-01-20T08:31:42.985042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'7","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981351+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:31:42.981351+0000","last_peered":"2024-01-20T08:31:42.981351+0000","last_clean":"2024-01-20T08:31:42.981351+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:31:42.981351+0000","last_undegraded":"2024-01-20T08:31:42.981351+0000","last_fullsized":"2024-01-20T08:31:42.981351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571548+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:31:19.571548+0000","last_peered":"2024-01-20T08:31:19.571548+0000","last_clean":"2024-01-20T08:31:19.571548+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:31:19.571548+0000","last_undegraded":"2024-01-20T08:31:19.571548+0000","last_fullsized":"2024-01-20T08:31:19.571548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978873+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:31:42.978873+0000","last_peered":"2024-01-20T08:31:42.978873+0000","last_clean":"2024-01-20T08:31:42.978873+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:31:42.978873+0000","last_undegraded":"2024-01-20T08:31:42.978873+0000","last_fullsized":"2024-01-20T08:31:42.978873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'14","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.808950+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:31:44.808950+0000","last_peered":"2024-01-20T08:31:44.808950+0000","last_clean":"2024-01-20T08:31:44.808950+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:31:44.808950+0000","last_undegraded":"2024-01-20T08:31:44.808950+0000","last_fullsized":"2024-01-20T08:31:44.808950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984558+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:31:42.984558+0000","last_peered":"2024-01-20T08:31:42.984558+0000","last_clean":"2024-01-20T08:31:42.984558+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:31:42.984558+0000","last_undegraded":"2024-01-20T08:31:42.984558+0000","last_fullsized":"2024-01-20T08:31:42.984558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136437+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:31:17.136437+0000","last_peered":"2024-01-20T08:31:17.136437+0000","last_clean":"2024-01-20T08:31:17.136437+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:31:17.136437+0000","last_undegraded":"2024-01-20T08:31:17.136437+0000","last_fullsized":"2024-01-20T08:31:17.136437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981879+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:31:42.981879+0000","last_peered":"2024-01-20T08:31:42.981879+0000","last_clean":"2024-01-20T08:31:42.981879+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:31:42.981879+0000","last_undegraded":"2024-01-20T08:31:42.981879+0000","last_fullsized":"2024-01-20T08:31:42.981879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244323+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:31:43.244323+0000","last_peered":"2024-01-20T08:31:43.244323+0000","last_clean":"2024-01-20T08:31:43.244323+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:31:43.244323+0000","last_undegraded":"2024-01-20T08:31:43.244323+0000","last_fullsized":"2024-01-20T08:31:43.244323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245075+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:31:43.245075+0000","last_peered":"2024-01-20T08:31:43.245075+0000","last_clean":"2024-01-20T08:31:43.245075+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:31:43.245075+0000","last_undegraded":"2024-01-20T08:31:43.245075+0000","last_fullsized":"2024-01-20T08:31:43.245075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.865858+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:31:44.865858+0000","last_peered":"2024-01-20T08:31:44.865858+0000","last_clean":"2024-01-20T08:31:44.865858+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:31:44.865858+0000","last_undegraded":"2024-01-20T08:31:44.865858+0000","last_fullsized":"2024-01-20T08:31:44.865858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.980966+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:31:42.980966+0000","last_peered":"2024-01-20T08:31:42.980966+0000","last_clean":"2024-01-20T08:31:42.980966+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:31:42.980966+0000","last_undegraded":"2024-01-20T08:31:42.980966+0000","last_fullsized":"2024-01-20T08:31:42.980966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244656+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:31:43.244656+0000","last_peered":"2024-01-20T08:31:43.244656+0000","last_clean":"2024-01-20T08:31:43.244656+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:31:43.244656+0000","last_undegraded":"2024-01-20T08:31:43.244656+0000","last_fullsized":"2024-01-20T08:31:43.244656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'4","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983703+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:31:42.983703+0000","last_peered":"2024-01-20T08:31:42.983703+0000","last_clean":"2024-01-20T08:31:42.983703+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:31:42.983703+0000","last_undegraded":"2024-01-20T08:31:42.983703+0000","last_fullsized":"2024-01-20T08:31:42.983703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985885+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:31:42.985885+0000","last_peered":"2024-01-20T08:31:42.985885+0000","last_clean":"2024-01-20T08:31:42.985885+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:31:42.985885+0000","last_undegraded":"2024-01-20T08:31:42.985885+0000","last_fullsized":"2024-01-20T08:31:42.985885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984122+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:31:42.984122+0000","last_peered":"2024-01-20T08:31:42.984122+0000","last_clean":"2024-01-20T08:31:42.984122+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:31:42.984122+0000","last_undegraded":"2024-01-20T08:31:42.984122+0000","last_fullsized":"2024-01-20T08:31:42.984122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985901+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:31:42.985901+0000","last_peered":"2024-01-20T08:31:42.985901+0000","last_clean":"2024-01-20T08:31:42.985901+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:31:42.985901+0000","last_undegraded":"2024-01-20T08:31:42.985901+0000","last_fullsized":"2024-01-20T08:31:42.985901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.603336+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:31:42.603336+0000","last_peered":"2024-01-20T08:31:42.603336+0000","last_clean":"2024-01-20T08:31:42.603336+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:31:42.603336+0000","last_undegraded":"2024-01-20T08:31:42.603336+0000","last_fullsized":"2024-01-20T08:31:42.603336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:19.571569+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:31:19.571569+0000","last_peered":"2024-01-20T08:31:19.571569+0000","last_clean":"2024-01-20T08:31:19.571569+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:31:19.571569+0000","last_undegraded":"2024-01-20T08:31:19.571569+0000","last_fullsized":"2024-01-20T08:31:19.571569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244726+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:31:43.244726+0000","last_peered":"2024-01-20T08:31:43.244726+0000","last_clean":"2024-01-20T08:31:43.244726+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:31:43.244726+0000","last_undegraded":"2024-01-20T08:31:43.244726+0000","last_fullsized":"2024-01-20T08:31:43.244726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:31:44.208858+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:31:44.208858+0000","last_peered":"2024-01-20T08:31:44.208858+0000","last_clean":"2024-01-20T08:31:44.208858+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:31:44.208858+0000","last_undegraded":"2024-01-20T08:31:44.208858+0000","last_fullsized":"2024-01-20T08:31:44.208858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'7","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.605951+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:31:42.605951+0000","last_peered":"2024-01-20T08:31:42.605951+0000","last_clean":"2024-01-20T08:31:42.605951+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:31:42.605951+0000","last_undegraded":"2024-01-20T08:31:42.605951+0000","last_fullsized":"2024-01-20T08:31:42.605951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984935+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:31:42.984935+0000","last_peered":"2024-01-20T08:31:42.984935+0000","last_clean":"2024-01-20T08:31:42.984935+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:31:42.984935+0000","last_undegraded":"2024-01-20T08:31:42.984935+0000","last_fullsized":"2024-01-20T08:31:42.984935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.985339+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:31:42.985339+0000","last_peered":"2024-01-20T08:31:42.985339+0000","last_clean":"2024-01-20T08:31:42.985339+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:31:42.985339+0000","last_undegraded":"2024-01-20T08:31:42.985339+0000","last_fullsized":"2024-01-20T08:31:42.985339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245102+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:31:43.245102+0000","last_peered":"2024-01-20T08:31:43.245102+0000","last_clean":"2024-01-20T08:31:43.245102+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:31:43.245102+0000","last_undegraded":"2024-01-20T08:31:43.245102+0000","last_fullsized":"2024-01-20T08:31:43.245102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983553+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:31:42.983553+0000","last_peered":"2024-01-20T08:31:42.983553+0000","last_clean":"2024-01-20T08:31:42.983553+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:31:42.983553+0000","last_undegraded":"2024-01-20T08:31:42.983553+0000","last_fullsized":"2024-01-20T08:31:42.983553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981680+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:31:42.981680+0000","last_peered":"2024-01-20T08:31:42.981680+0000","last_clean":"2024-01-20T08:31:42.981680+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:31:42.981680+0000","last_undegraded":"2024-01-20T08:31:42.981680+0000","last_fullsized":"2024-01-20T08:31:42.981680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244377+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:31:43.244377+0000","last_peered":"2024-01-20T08:31:43.244377+0000","last_clean":"2024-01-20T08:31:43.244377+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:31:43.244377+0000","last_undegraded":"2024-01-20T08:31:43.244377+0000","last_fullsized":"2024-01-20T08:31:43.244377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981454+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:31:42.981454+0000","last_peered":"2024-01-20T08:31:42.981454+0000","last_clean":"2024-01-20T08:31:42.981454+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:31:42.981454+0000","last_undegraded":"2024-01-20T08:31:42.981454+0000","last_fullsized":"2024-01-20T08:31:42.981454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136499+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:31:17.136499+0000","last_peered":"2024-01-20T08:31:17.136499+0000","last_clean":"2024-01-20T08:31:17.136499+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:31:17.136499+0000","last_undegraded":"2024-01-20T08:31:17.136499+0000","last_fullsized":"2024-01-20T08:31:17.136499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.245061+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:31:43.245061+0000","last_peered":"2024-01-20T08:31:43.245061+0000","last_clean":"2024-01-20T08:31:43.245061+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:31:43.245061+0000","last_undegraded":"2024-01-20T08:31:43.245061+0000","last_fullsized":"2024-01-20T08:31:43.245061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984759+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:31:42.984759+0000","last_peered":"2024-01-20T08:31:42.984759+0000","last_clean":"2024-01-20T08:31:42.984759+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:31:42.984759+0000","last_undegraded":"2024-01-20T08:31:42.984759+0000","last_fullsized":"2024-01-20T08:31:42.984759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984413+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:31:42.984413+0000","last_peered":"2024-01-20T08:31:42.984413+0000","last_clean":"2024-01-20T08:31:42.984413+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:31:42.984413+0000","last_undegraded":"2024-01-20T08:31:42.984413+0000","last_fullsized":"2024-01-20T08:31:42.984413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244577+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:31:43.244577+0000","last_peered":"2024-01-20T08:31:43.244577+0000","last_clean":"2024-01-20T08:31:43.244577+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:31:43.244577+0000","last_undegraded":"2024-01-20T08:31:43.244577+0000","last_fullsized":"2024-01-20T08:31:43.244577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244973+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:31:43.244973+0000","last_peered":"2024-01-20T08:31:43.244973+0000","last_clean":"2024-01-20T08:31:43.244973+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:31:43.244973+0000","last_undegraded":"2024-01-20T08:31:43.244973+0000","last_fullsized":"2024-01-20T08:31:43.244973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136221+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:31:17.136221+0000","last_peered":"2024-01-20T08:31:17.136221+0000","last_clean":"2024-01-20T08:31:17.136221+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:31:17.136221+0000","last_undegraded":"2024-01-20T08:31:17.136221+0000","last_fullsized":"2024-01-20T08:31:17.136221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244645+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:31:43.244645+0000","last_peered":"2024-01-20T08:31:43.244645+0000","last_clean":"2024-01-20T08:31:43.244645+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:31:43.244645+0000","last_undegraded":"2024-01-20T08:31:43.244645+0000","last_fullsized":"2024-01-20T08:31:43.244645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983059+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:31:42.983059+0000","last_peered":"2024-01-20T08:31:42.983059+0000","last_clean":"2024-01-20T08:31:42.983059+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:31:42.983059+0000","last_undegraded":"2024-01-20T08:31:42.983059+0000","last_fullsized":"2024-01-20T08:31:42.983059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984863+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:31:42.984863+0000","last_peered":"2024-01-20T08:31:42.984863+0000","last_clean":"2024-01-20T08:31:42.984863+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:31:42.984863+0000","last_undegraded":"2024-01-20T08:31:42.984863+0000","last_fullsized":"2024-01-20T08:31:42.984863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978831+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:31:42.978831+0000","last_peered":"2024-01-20T08:31:42.978831+0000","last_clean":"2024-01-20T08:31:42.978831+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:31:42.978831+0000","last_undegraded":"2024-01-20T08:31:42.978831+0000","last_fullsized":"2024-01-20T08:31:42.978831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.137287+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:31:17.137287+0000","last_peered":"2024-01-20T08:31:17.137287+0000","last_clean":"2024-01-20T08:31:17.137287+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:31:17.137287+0000","last_undegraded":"2024-01-20T08:31:17.137287+0000","last_fullsized":"2024-01-20T08:31:17.137287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.981474+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:31:42.981474+0000","last_peered":"2024-01-20T08:31:42.981474+0000","last_clean":"2024-01-20T08:31:42.981474+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:31:42.981474+0000","last_undegraded":"2024-01-20T08:31:42.981474+0000","last_fullsized":"2024-01-20T08:31:42.981474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136720+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:31:17.136720+0000","last_peered":"2024-01-20T08:31:17.136720+0000","last_clean":"2024-01-20T08:31:17.136720+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:31:17.136720+0000","last_undegraded":"2024-01-20T08:31:17.136720+0000","last_fullsized":"2024-01-20T08:31:17.136720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-20T08:31:17.136207+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:31:17.136207+0000","last_peered":"2024-01-20T08:31:17.136207+0000","last_clean":"2024-01-20T08:31:17.136207+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:31:17.136207+0000","last_undegraded":"2024-01-20T08:31:17.136207+0000","last_fullsized":"2024-01-20T08:31:17.136207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244598+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:31:43.244598+0000","last_peered":"2024-01-20T08:31:43.244598+0000","last_clean":"2024-01-20T08:31:43.244598+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:31:43.244598+0000","last_undegraded":"2024-01-20T08:31:43.244598+0000","last_fullsized":"2024-01-20T08:31:43.244598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984971+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:31:42.984971+0000","last_peered":"2024-01-20T08:31:42.984971+0000","last_clean":"2024-01-20T08:31:42.984971+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:31:42.984971+0000","last_undegraded":"2024-01-20T08:31:42.984971+0000","last_fullsized":"2024-01-20T08:31:42.984971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.984181+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:31:42.984181+0000","last_peered":"2024-01-20T08:31:42.984181+0000","last_clean":"2024-01-20T08:31:42.984181+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:31:42.984181+0000","last_undegraded":"2024-01-20T08:31:42.984181+0000","last_fullsized":"2024-01-20T08:31:42.984181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978794+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:31:42.978794+0000","last_peered":"2024-01-20T08:31:42.978794+0000","last_clean":"2024-01-20T08:31:42.978794+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:31:42.978794+0000","last_undegraded":"2024-01-20T08:31:42.978794+0000","last_fullsized":"2024-01-20T08:31:42.978794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.979087+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:31:42.979087+0000","last_peered":"2024-01-20T08:31:42.979087+0000","last_clean":"2024-01-20T08:31:42.979087+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:31:42.979087+0000","last_undegraded":"2024-01-20T08:31:42.979087+0000","last_fullsized":"2024-01-20T08:31:42.979087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978943+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:31:42.978943+0000","last_peered":"2024-01-20T08:31:42.978943+0000","last_clean":"2024-01-20T08:31:42.978943+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:31:42.978943+0000","last_undegraded":"2024-01-20T08:31:42.978943+0000","last_fullsized":"2024-01-20T08:31:42.978943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.983499+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:31:42.983499+0000","last_peered":"2024-01-20T08:31:42.983499+0000","last_clean":"2024-01-20T08:31:42.983499+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:31:42.983499+0000","last_undegraded":"2024-01-20T08:31:42.983499+0000","last_fullsized":"2024-01-20T08:31:42.983499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244621+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:31:43.244621+0000","last_peered":"2024-01-20T08:31:43.244621+0000","last_clean":"2024-01-20T08:31:43.244621+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:31:43.244621+0000","last_undegraded":"2024-01-20T08:31:43.244621+0000","last_fullsized":"2024-01-20T08:31:43.244621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978481+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:31:42.978481+0000","last_peered":"2024-01-20T08:31:42.978481+0000","last_clean":"2024-01-20T08:31:42.978481+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:31:42.978481+0000","last_undegraded":"2024-01-20T08:31:42.978481+0000","last_fullsized":"2024-01-20T08:31:42.978481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244425+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:31:43.244425+0000","last_peered":"2024-01-20T08:31:43.244425+0000","last_clean":"2024-01-20T08:31:43.244425+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:31:43.244425+0000","last_undegraded":"2024-01-20T08:31:43.244425+0000","last_fullsized":"2024-01-20T08:31:43.244425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978643+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:31:42.978643+0000","last_peered":"2024-01-20T08:31:42.978643+0000","last_clean":"2024-01-20T08:31:42.978643+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:31:42.978643+0000","last_undegraded":"2024-01-20T08:31:42.978643+0000","last_fullsized":"2024-01-20T08:31:42.978643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244404+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:31:43.244404+0000","last_peered":"2024-01-20T08:31:43.244404+0000","last_clean":"2024-01-20T08:31:43.244404+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:31:43.244404+0000","last_undegraded":"2024-01-20T08:31:43.244404+0000","last_fullsized":"2024-01-20T08:31:43.244404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244863+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:31:43.244863+0000","last_peered":"2024-01-20T08:31:43.244863+0000","last_clean":"2024-01-20T08:31:43.244863+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:31:43.244863+0000","last_undegraded":"2024-01-20T08:31:43.244863+0000","last_fullsized":"2024-01-20T08:31:43.244863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:43.244967+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:31:43.244967+0000","last_peered":"2024-01-20T08:31:43.244967+0000","last_clean":"2024-01-20T08:31:43.244967+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:31:43.244967+0000","last_undegraded":"2024-01-20T08:31:43.244967+0000","last_fullsized":"2024-01-20T08:31:43.244967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-20T08:31:42.978956+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:31:42.978956+0000","last_peered":"2024-01-20T08:31:42.978956+0000","last_clean":"2024-01-20T08:31:42.978956+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:31:42.978956+0000","last_undegraded":"2024-01-20T08:31:42.978956+0000","last_fullsized":"2024-01-20T08:31:42.978956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9529,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":19702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168,"ondisk_log_size":168,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":119853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":708667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":123152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":121382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":709255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":319488,"data_stored":118505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:31:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":112659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:31:49.972 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-20T08:31:49.972 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-20T08:31:49.972 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-20T08:31:49.973 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-20T08:31:50.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.102+0000 7f213844f640 1 Processor -- start 2024-01-20T08:31:50.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.102+0000 7f213844f640 1 -- start start 2024-01-20T08:31:50.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.103+0000 7f213844f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2130153410 0x7f21300ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.103+0000 7f213844f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f21301537f0 0x7f2130159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.103+0000 7f213844f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213015cb40 con 0x7f213006a5b0 2024-01-20T08:31:50.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.103+0000 7f213844f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213015a510 con 0x7f2130153410 2024-01-20T08:31:50.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.103+0000 7f213844f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213015d9a0 con 0x7f21301537f0 2024-01-20T08:31:50.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.104+0000 7f21361eb640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2130153410 0x7f21300ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.104+0000 7f21361eb640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2130153410 0x7f21300ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45772/0 (socket says 172.21.15.87:45772) 2024-01-20T08:31:50.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.104+0000 7f21361eb640 1 -- 172.21.15.87:0/757268393 learned_addr learned my addr 172.21.15.87:0/757268393 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:50.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.105+0000 7f21359ea640 1 --2- 172.21.15.87:0/757268393 >> v2:172.21.15.116:3300/0 conn(0x7f21301537f0 0x7f2130159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.107+0000 7f21361eb640 1 -- 172.21.15.87:0/757268393 >> v2:172.21.15.116:3300/0 conn(0x7f21301537f0 msgr2=0x7f2130159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.107+0000 7f21361eb640 1 --2- 172.21.15.87:0/757268393 >> v2:172.21.15.116:3300/0 conn(0x7f21301537f0 0x7f2130159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.107+0000 7f21361eb640 1 -- 172.21.15.87:0/757268393 >> v1:172.21.15.116:6789/0 conn(0x7f213006a5b0 legacy=0x7f21300c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.107+0000 7f21361eb640 1 -- 172.21.15.87:0/757268393 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213015cd90 con 0x7f2130153410 2024-01-20T08:31:50.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.109+0000 7f21361eb640 1 --2- 172.21.15.87:0/757268393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2130153410 0x7f21300ca230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f213006fc00 tx=0x7f212c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce037bdad566421d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:50.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.112+0000 7f21351e9640 1 -- 172.21.15.87:0/757268393 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c02f850 con 0x7f2130153410 2024-01-20T08:31:50.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.112+0000 7f21351e9640 1 -- 172.21.15.87:0/757268393 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c02f9b0 con 0x7f2130153410 2024-01-20T08:31:50.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.112+0000 7f21351e9640 1 -- 172.21.15.87:0/757268393 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c02fd40 con 0x7f2130153410 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 -- 172.21.15.87:0/757268393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2130153410 msgr2=0x7f21300ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 --2- 172.21.15.87:0/757268393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2130153410 0x7f21300ca230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f213006fc00 tx=0x7f212c02ba30 comp rx=0 tx=0).stop 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 -- 172.21.15.87:0/757268393 shutdown_connections 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 --2- 172.21.15.87:0/757268393 >> v2:172.21.15.116:3300/0 conn(0x7f21301537f0 0x7f2130159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 --2- 172.21.15.87:0/757268393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2130153410 0x7f21300ca230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 -- 172.21.15.87:0/757268393 >> 172.21.15.87:0/757268393 conn(0x7f213006eeb0 msgr2=0x7f213006f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 -- 172.21.15.87:0/757268393 shutdown_connections 2024-01-20T08:31:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.113+0000 7f213844f640 1 -- 172.21.15.87:0/757268393 wait complete. 2024-01-20T08:31:50.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.114+0000 7f213844f640 1 Processor -- start 2024-01-20T08:31:50.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.114+0000 7f213844f640 1 -- start start 2024-01-20T08:31:50.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.115+0000 7f213844f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 0x7f21300cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.115+0000 7f213844f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2130153410 0x7f21300b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.115+0000 7f213844f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21301537f0 0x7f21300cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.115+0000 7f213844f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21300b1a90 con 0x7f213006a5b0 2024-01-20T08:31:50.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.115+0000 7f213844f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21301f33d0 con 0x7f21301537f0 2024-01-20T08:31:50.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.115+0000 7f213844f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21301f2f40 con 0x7f2130153410 2024-01-20T08:31:50.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.116+0000 7f21361eb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 0x7f21300cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.116+0000 7f21369ec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21301537f0 0x7f21300cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.116+0000 7f21361eb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 0x7f21300cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45780/0 (socket says 172.21.15.87:45780) 2024-01-20T08:31:50.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.116+0000 7f21361eb640 1 -- 172.21.15.87:0/3322690029 learned_addr learned my addr 172.21.15.87:0/3322690029 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:50.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.116+0000 7f21359ea640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2130153410 0x7f21300b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.119+0000 7f21361eb640 1 -- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2130153410 msgr2=0x7f21300b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.119+0000 7f21361eb640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2130153410 0x7f21300b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.119+0000 7f21361eb640 1 -- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21301537f0 msgr2=0x7f21300cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.119+0000 7f21361eb640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21301537f0 0x7f21300cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.119+0000 7f21361eb640 1 -- 172.21.15.87:0/3322690029 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c0094d0 con 0x7f213006a5b0 2024-01-20T08:31:50.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.121+0000 7f21361eb640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 0x7f21300cc9e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f212c009960 tx=0x7f212c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.124+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c03b040 con 0x7f213006a5b0 2024-01-20T08:31:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.125+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c037c60 con 0x7f213006a5b0 2024-01-20T08:31:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.125+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c042460 con 0x7f213006a5b0 2024-01-20T08:31:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.125+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21300b17b0 con 0x7f213006a5b0 2024-01-20T08:31:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.125+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21301f28f0 con 0x7f213006a5b0 2024-01-20T08:31:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.126+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21300b3180 con 0x7f213006a5b0 2024-01-20T08:31:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.367+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f212c03f3f0 con 0x7f213006a5b0 2024-01-20T08:31:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.368+0000 7f2122ffd640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211406af20 0x7f211406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.368+0000 7f21359ea640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211406af20 0x7f211406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.368+0000 7f21359ea640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211406af20 0x7f211406d3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21301e3f30 tx=0x7f2124009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.393+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f212c07aec0 con 0x7f213006a5b0 2024-01-20T08:31:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.393+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f212c077200 con 0x7f213006a5b0 2024-01-20T08:31:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.605+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2130074290 con 0x7f213006a5b0 2024-01-20T08:31:50.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.656+0000 7f2122ffd640 1 -- 172.21.15.87:0/3322690029 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f212c047070 con 0x7f213006a5b0 2024-01-20T08:31:50.660 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:50.660 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-20T08:31:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211406af20 msgr2=0x7f211406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211406af20 0x7f211406d3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21301e3f30 tx=0x7f2124009190 comp rx=0 tx=0).stop 2024-01-20T08:31:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 msgr2=0x7f21300cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 0x7f21300cc9e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f212c009960 tx=0x7f212c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 shutdown_connections 2024-01-20T08:31:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f211406af20 0x7f211406d3e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21301537f0 0x7f21300cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2130153410 0x7f21300b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 --2- 172.21.15.87:0/3322690029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f213006a5b0 0x7f21300cc9e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 >> 172.21.15.87:0/3322690029 conn(0x7f213006eeb0 msgr2=0x7f213015b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 shutdown_connections 2024-01-20T08:31:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.659+0000 7f213844f640 1 -- 172.21.15.87:0/3322690029 wait complete. 2024-01-20T08:31:50.672 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-20T08:31:50.673 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:50.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.771+0000 7fa320f57640 1 Processor -- start 2024-01-20T08:31:50.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.771+0000 7fa320f57640 1 -- start start 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa320f57640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa31c153410 0x7fa31c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa320f57640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa31c159fd0 0x7fa31c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa320f57640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c15cb40 con 0x7fa31c1537f0 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa320f57640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c15d9a0 con 0x7fa31c153410 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa320f57640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c15cd90 con 0x7fa31c159fd0 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa31a59c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa31c153410 0x7fa31c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.772+0000 7fa31ad9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa31c159fd0 0x7fa31c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.773+0000 7fa313fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa31c1537f0 0x7fa31c06a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:36470/0 (socket says 172.21.15.87:36470) 2024-01-20T08:31:50.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.773+0000 7fa313fff640 1 -- 172.21.15.87:0/3266768501 learned_addr learned my addr 172.21.15.87:0/3266768501 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:50.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.775+0000 7fa31a59c640 1 -- 172.21.15.87:0/3266768501 >> v2:172.21.15.116:3300/0 conn(0x7fa31c159fd0 msgr2=0x7fa31c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:50.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.775+0000 7fa31a59c640 1 --2- 172.21.15.87:0/3266768501 >> v2:172.21.15.116:3300/0 conn(0x7fa31c159fd0 0x7fa31c0c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.775+0000 7fa31a59c640 1 -- 172.21.15.87:0/3266768501 >> v1:172.21.15.116:6789/0 conn(0x7fa31c1537f0 legacy=0x7fa31c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.775+0000 7fa31a59c640 1 -- 172.21.15.87:0/3266768501 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c15d070 con 0x7fa31c153410 2024-01-20T08:31:50.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.777+0000 7fa31a59c640 1 --2- 172.21.15.87:0/3266768501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c0ca230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa31c06b560 tx=0x7fa30c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e905a79b406a84f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:50.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.780+0000 7fa319d9b640 1 -- 172.21.15.87:0/3266768501 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c030610 con 0x7fa31c153410 2024-01-20T08:31:50.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.780+0000 7fa319d9b640 1 -- 172.21.15.87:0/3266768501 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c030770 con 0x7fa31c153410 2024-01-20T08:31:50.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.780+0000 7fa319d9b640 1 -- 172.21.15.87:0/3266768501 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c030b00 con 0x7fa31c153410 2024-01-20T08:31:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 -- 172.21.15.87:0/3266768501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 msgr2=0x7fa31c0ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 --2- 172.21.15.87:0/3266768501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c0ca230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa31c06b560 tx=0x7fa30c02b960 comp rx=0 tx=0).stop 2024-01-20T08:31:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 -- 172.21.15.87:0/3266768501 shutdown_connections 2024-01-20T08:31:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 --2- 172.21.15.87:0/3266768501 >> v2:172.21.15.116:3300/0 conn(0x7fa31c159fd0 0x7fa31c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 --2- 172.21.15.87:0/3266768501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c0ca230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 -- 172.21.15.87:0/3266768501 >> 172.21.15.87:0/3266768501 conn(0x7fa31c06eeb0 msgr2=0x7fa31c06f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:50.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 -- 172.21.15.87:0/3266768501 shutdown_connections 2024-01-20T08:31:50.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.781+0000 7fa320f57640 1 -- 172.21.15.87:0/3266768501 wait complete. 2024-01-20T08:31:50.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.782+0000 7fa320f57640 1 Processor -- start 2024-01-20T08:31:50.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.782+0000 7fa320f57640 1 -- start start 2024-01-20T08:31:50.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa320f57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa320f57640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa31c1537f0 0x7fa31c1ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa320f57640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa31c159fd0 0x7fa31c14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa320f57640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0cd7b0 con 0x7fa31c153410 2024-01-20T08:31:50.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa320f57640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0cad60 con 0x7fa31c159fd0 2024-01-20T08:31:50.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa320f57640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1e2a10 con 0x7fa31c1537f0 2024-01-20T08:31:50.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.783+0000 7fa31a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.784+0000 7fa31ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa31c159fd0 0x7fa31c14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.784+0000 7fa31a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c1e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45802/0 (socket says 172.21.15.87:45802) 2024-01-20T08:31:50.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.784+0000 7fa31a59c640 1 -- 172.21.15.87:0/2579254125 learned_addr learned my addr 172.21.15.87:0/2579254125 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:50.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.784+0000 7fa313fff640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa31c1537f0 0x7fa31c1ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.787+0000 7fa31ad9d640 1 -- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa31c1537f0 msgr2=0x7fa31c1ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.787+0000 7fa31ad9d640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa31c1537f0 0x7fa31c1ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.787+0000 7fa31ad9d640 1 -- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 msgr2=0x7fa31c1e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:50.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.787+0000 7fa31ad9d640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c1e5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:50.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.787+0000 7fa31ad9d640 1 -- 172.21.15.87:0/2579254125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009400 con 0x7fa31c159fd0 2024-01-20T08:31:50.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.789+0000 7fa31ad9d640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa31c159fd0 0x7fa31c14e960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa31400eb80 tx=0x7fa31400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:50.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.793+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31400cd20 con 0x7fa31c159fd0 2024-01-20T08:31:50.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.793+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31400f040 con 0x7fa31c159fd0 2024-01-20T08:31:50.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.793+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31400cea0 con 0x7fa31c159fd0 2024-01-20T08:31:50.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.793+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c0cda50 con 0x7fa31c159fd0 2024-01-20T08:31:50.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.793+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c1e3050 con 0x7fa31c159fd0 2024-01-20T08:31:50.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.794+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c0caf20 con 0x7fa31c159fd0 2024-01-20T08:31:50.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.813+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa3140146c0 con 0x7fa31c159fd0 2024-01-20T08:31:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.814+0000 7fa312ffd640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2fc06af20 0x7fa2fc06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.814+0000 7fa31a59c640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2fc06af20 0x7fa2fc06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:50.814+0000 7fa31a59c640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2fc06af20 0x7fa2fc06d3e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa30c02b9a0 tx=0x7fa30c008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.039+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fa314059350 con 0x7fa31c159fd0 2024-01-20T08:31:51.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.049+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa314055690 con 0x7fa31c159fd0 2024-01-20T08:31:51.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.219+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa31c074290 con 0x7fa31c159fd0 2024-01-20T08:31:51.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.255+0000 7fa312ffd640 1 -- 172.21.15.87:0/2579254125 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7fa314018070 con 0x7fa31c159fd0 2024-01-20T08:31:51.258 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:51.258 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:51.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2fc06af20 msgr2=0x7fa2fc06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2fc06af20 0x7fa2fc06d3e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa30c02b9a0 tx=0x7fa30c008030 comp rx=0 tx=0).stop 2024-01-20T08:31:51.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa31c159fd0 msgr2=0x7fa31c14e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa31c159fd0 0x7fa31c14e960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa31400eb80 tx=0x7fa31400c490 comp rx=0 tx=0).stop 2024-01-20T08:31:51.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 shutdown_connections 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa2fc06af20 0x7fa2fc06d3e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa31c159fd0 0x7fa31c14e960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa31c1537f0 0x7fa31c1ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 --2- 172.21.15.87:0/2579254125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa31c153410 0x7fa31c1e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 >> 172.21.15.87:0/2579254125 conn(0x7fa31c06eeb0 msgr2=0x7fa31c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 shutdown_connections 2024-01-20T08:31:51.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.257+0000 7fa320f57640 1 -- 172.21.15.87:0/2579254125 wait complete. 2024-01-20T08:31:51.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-20T08:31:06.500533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-20T08:31:08.175669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-20T08:31:13.903374+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-20T08:31:51.272 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:31:51.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.372+0000 7fa3ac9d7640 1 Processor -- start 2024-01-20T08:31:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.372+0000 7fa3ac9d7640 1 -- start start 2024-01-20T08:31:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.373+0000 7fa3ac9d7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa3a80bfe70 0x7fa3a80c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.373+0000 7fa3ac9d7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a8156720 con 0x7fa3a814cc40 2024-01-20T08:31:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.373+0000 7fa3ac9d7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a80c19c0 con 0x7fa3a80c81f0 2024-01-20T08:31:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.373+0000 7fa3ac9d7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a81572c0 con 0x7fa3a80bfe70 2024-01-20T08:31:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.374+0000 7fa3a659c640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa3a814cc40 0x7fa3a8156bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33580/0 (socket says 172.21.15.87:33580) 2024-01-20T08:31:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.374+0000 7fa3a659c640 1 -- 172.21.15.87:0/374734504 learned_addr learned my addr 172.21.15.87:0/374734504 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.374+0000 7fa3a5d9b640 1 --2- 172.21.15.87:0/374734504 >> v2:172.21.15.116:3300/0 conn(0x7fa3a80bfe70 0x7fa3a80c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.376+0000 7fa3a559a640 1 -- 172.21.15.87:0/374734504 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635696042 0 0) 0x7fa3a8156720 con 0x7fa3a814cc40 2024-01-20T08:31:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.376+0000 7fa3a559a640 1 -- 172.21.15.87:0/374734504 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa390005730 con 0x7fa3a814cc40 2024-01-20T08:31:51.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.377+0000 7fa3a5d9b640 1 -- 172.21.15.87:0/374734504 >> v1:172.21.15.116:6789/0 conn(0x7fa3a80c81f0 legacy=0x7fa3a8160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.377+0000 7fa3a5d9b640 1 -- 172.21.15.87:0/374734504 >> v1:172.21.15.87:6789/0 conn(0x7fa3a814cc40 legacy=0x7fa3a8156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.377+0000 7fa3a5d9b640 1 -- 172.21.15.87:0/374734504 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a81563f0 con 0x7fa3a80bfe70 2024-01-20T08:31:51.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.380+0000 7fa3a5d9b640 1 --2- 172.21.15.87:0/374734504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c7cb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0097a0 tx=0x7fa39c02b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b33eedb903f1428 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.383+0000 7fa3a559a640 1 -- 172.21.15.87:0/374734504 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c02f810 con 0x7fa3a80bfe70 2024-01-20T08:31:51.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.384+0000 7fa3a559a640 1 -- 172.21.15.87:0/374734504 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c02f970 con 0x7fa3a80bfe70 2024-01-20T08:31:51.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.384+0000 7fa3a559a640 1 -- 172.21.15.87:0/374734504 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c02fce0 con 0x7fa3a80bfe70 2024-01-20T08:31:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/374734504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 msgr2=0x7fa3a80c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/374734504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c7cb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0097a0 tx=0x7fa39c02b9f0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/374734504 shutdown_connections 2024-01-20T08:31:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/374734504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c7cb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/374734504 >> 172.21.15.87:0/374734504 conn(0x7fa3a806a9a0 msgr2=0x7fa3a806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:51.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/374734504 shutdown_connections 2024-01-20T08:31:51.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.385+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/374734504 wait complete. 2024-01-20T08:31:51.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.386+0000 7fa3ac9d7640 1 Processor -- start 2024-01-20T08:31:51.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 -- start start 2024-01-20T08:31:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3a80c81f0 0x7fa3a80c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3a814cc40 0x7fa3a80c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a80c20c0 con 0x7fa3a80c81f0 2024-01-20T08:31:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a806b270 con 0x7fa3a80bfe70 2024-01-20T08:31:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.387+0000 7fa3ac9d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a80cb570 con 0x7fa3a814cc40 2024-01-20T08:31:51.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.388+0000 7fa3a659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.388+0000 7fa3a5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3a80c81f0 0x7fa3a80c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.388+0000 7fa3a6d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3a814cc40 0x7fa3a80c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.388+0000 7fa3a659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46680/0 (socket says 172.21.15.87:46680) 2024-01-20T08:31:51.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.388+0000 7fa3a659c640 1 -- 172.21.15.87:0/212351848 learned_addr learned my addr 172.21.15.87:0/212351848 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:51.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.391+0000 7fa3a5d9b640 1 -- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3a814cc40 msgr2=0x7fa3a80c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.391+0000 7fa3a5d9b640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3a814cc40 0x7fa3a80c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.391+0000 7fa3a5d9b640 1 -- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 msgr2=0x7fa3a80c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.391+0000 7fa3a5d9b640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.391+0000 7fa3a5d9b640 1 -- 172.21.15.87:0/212351848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c009400 con 0x7fa3a80c81f0 2024-01-20T08:31:51.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.391+0000 7fa3a659c640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.393+0000 7fa3a5d9b640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3a80c81f0 0x7fa3a80c4650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa39c033000 tx=0x7fa39c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.397+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c03d070 con 0x7fa3a80c81f0 2024-01-20T08:31:51.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.397+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c004250 con 0x7fa3a80c81f0 2024-01-20T08:31:51.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.397+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a80cb810 con 0x7fa3a80c81f0 2024-01-20T08:31:51.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.397+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003610 con 0x7fa3a80c81f0 2024-01-20T08:31:51.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.397+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a80c25b0 con 0x7fa3a80c81f0 2024-01-20T08:31:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.398+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a80c5b10 con 0x7fa3a80c81f0 2024-01-20T08:31:51.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.416+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa39c003770 con 0x7fa3a80c81f0 2024-01-20T08:31:51.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.417+0000 7fa396ffd640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa38806af50 0x7fa38806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.417+0000 7fa3a659c640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa38806af50 0x7fa38806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.417+0000 7fa3a659c640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa38806af50 0x7fa38806d410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa398001560 tx=0x7fa398009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.649+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fa39c079de0 con 0x7fa3a80c81f0 2024-01-20T08:31:51.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.649+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa39c076120 con 0x7fa3a80c81f0 2024-01-20T08:31:51.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.824+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3a8074290 con 0x7fa3a80c81f0 2024-01-20T08:31:51.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.829+0000 7fa396ffd640 1 -- 172.21.15.87:0/212351848 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa39c046020 con 0x7fa3a80c81f0 2024-01-20T08:31:51.832 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa38806af50 msgr2=0x7fa38806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa38806af50 0x7fa38806d410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa398001560 tx=0x7fa398009040 comp rx=0 tx=0).stop 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3a80c81f0 msgr2=0x7fa3a80c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3a80c81f0 0x7fa3a80c4650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa39c033000 tx=0x7fa39c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 shutdown_connections 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa38806af50 0x7fa38806d410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3a814cc40 0x7fa3a80c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3a80c81f0 0x7fa3a80c4650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 --2- 172.21.15.87:0/212351848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3a80bfe70 0x7fa3a80c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 >> 172.21.15.87:0/212351848 conn(0x7fa3a806a9a0 msgr2=0x7fa3a81548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:51.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 shutdown_connections 2024-01-20T08:31:51.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.831+0000 7fa3ac9d7640 1 -- 172.21.15.87:0/212351848 wait complete. 2024-01-20T08:31:51.846 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-20T08:31:51.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.945+0000 7f2abbbcb640 1 Processor -- start 2024-01-20T08:31:51.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.945+0000 7f2abbbcb640 1 -- start start 2024-01-20T08:31:51.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.946+0000 7f2abbbcb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ab41539c0 0x7f2ab40c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.946+0000 7f2abbbcb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab40c90a0 0x7f2ab40c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.946+0000 7f2abbbcb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab40c7f50 con 0x7f2ab40c9f50 2024-01-20T08:31:51.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.946+0000 7f2abbbcb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab40c3c60 con 0x7f2ab41539c0 2024-01-20T08:31:51.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.946+0000 7f2abbbcb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab415cb10 con 0x7f2ab40c90a0 2024-01-20T08:31:51.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.946+0000 7f2ab9166640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab40c90a0 0x7f2ab40c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.947+0000 7f2aba168640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2ab40c9f50 0x7f2ab406edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:36488/0 (socket says 172.21.15.87:36488) 2024-01-20T08:31:51.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.947+0000 7f2aba168640 1 -- 172.21.15.87:0/1840764052 learned_addr learned my addr 172.21.15.87:0/1840764052 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:51.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.947+0000 7f2ab9967640 1 --2- 172.21.15.87:0/1840764052 >> v2:172.21.15.116:3300/0 conn(0x7f2ab41539c0 0x7f2ab40c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.949+0000 7f2ab9166640 1 -- 172.21.15.87:0/1840764052 >> v2:172.21.15.116:3300/0 conn(0x7f2ab41539c0 msgr2=0x7f2ab40c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.949+0000 7f2ab9166640 1 --2- 172.21.15.87:0/1840764052 >> v2:172.21.15.116:3300/0 conn(0x7f2ab41539c0 0x7f2ab40c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.949+0000 7f2ab9166640 1 -- 172.21.15.87:0/1840764052 >> v1:172.21.15.116:6789/0 conn(0x7f2ab40c9f50 legacy=0x7f2ab406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.949+0000 7f2ab9166640 1 -- 172.21.15.87:0/1840764052 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab415cd20 con 0x7f2ab40c90a0 2024-01-20T08:31:51.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.951+0000 7f2ab9166640 1 --2- 172.21.15.87:0/1840764052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab40c90a0 0x7f2ab40c9a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40097a0 tx=0x7f2aa402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9983273f7d013fdd server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.953+0000 7f2ab8965640 1 -- 172.21.15.87:0/1840764052 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4030610 con 0x7f2ab40c90a0 2024-01-20T08:31:51.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2ab8965640 1 -- 172.21.15.87:0/1840764052 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4030770 con 0x7f2ab40c90a0 2024-01-20T08:31:51.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2ab8965640 1 -- 172.21.15.87:0/1840764052 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4030b00 con 0x7f2ab40c90a0 2024-01-20T08:31:51.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 -- 172.21.15.87:0/1840764052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab40c90a0 msgr2=0x7f2ab40c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/1840764052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab40c90a0 0x7f2ab40c9a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40097a0 tx=0x7f2aa402b960 comp rx=0 tx=0).stop 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 -- 172.21.15.87:0/1840764052 shutdown_connections 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/1840764052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab40c90a0 0x7f2ab40c9a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/1840764052 >> v2:172.21.15.116:3300/0 conn(0x7f2ab41539c0 0x7f2ab40c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 -- 172.21.15.87:0/1840764052 >> 172.21.15.87:0/1840764052 conn(0x7f2ab4070de0 msgr2=0x7f2ab40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 -- 172.21.15.87:0/1840764052 shutdown_connections 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.955+0000 7f2abbbcb640 1 -- 172.21.15.87:0/1840764052 wait complete. 2024-01-20T08:31:51.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.956+0000 7f2abbbcb640 1 Processor -- start 2024-01-20T08:31:51.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 -- start start 2024-01-20T08:31:51.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ab40c90a0 0x7f2ab406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40c9f50 0x7f2ab41542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 0x7f2ab4148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4158140 con 0x7f2ab40c90a0 2024-01-20T08:31:51.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab415baf0 con 0x7f2ab40c9f50 2024-01-20T08:31:51.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.957+0000 7f2abbbcb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4158a20 con 0x7f2ab41539c0 2024-01-20T08:31:51.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.958+0000 7f2aba168640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 0x7f2ab4148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.958+0000 7f2ab9166640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40c9f50 0x7f2ab41542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.958+0000 7f2aba168640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 0x7f2ab4148470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47428/0 (socket says 172.21.15.87:47428) 2024-01-20T08:31:51.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.958+0000 7f2aba168640 1 -- 172.21.15.87:0/183146553 learned_addr learned my addr 172.21.15.87:0/183146553 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:51.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.958+0000 7f2ab9967640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ab40c90a0 0x7f2ab406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.961+0000 7f2aba168640 1 -- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40c9f50 msgr2=0x7f2ab41542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.961+0000 7f2aba168640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40c9f50 0x7f2ab41542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.961+0000 7f2aba168640 1 -- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ab40c90a0 msgr2=0x7f2ab406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.961+0000 7f2aba168640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ab40c90a0 0x7f2ab406b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.961+0000 7f2aba168640 1 -- 172.21.15.87:0/183146553 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4009400 con 0x7f2ab41539c0 2024-01-20T08:31:51.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.964+0000 7f2aba168640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 0x7f2ab4148470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00cab0 tx=0x7f2aac00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.967+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00ac30 con 0x7f2ab41539c0 2024-01-20T08:31:51.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.967+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00ad90 con 0x7f2ab41539c0 2024-01-20T08:31:51.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.967+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab41e4080 con 0x7f2ab41539c0 2024-01-20T08:31:51.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.967+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac012600 con 0x7f2ab41539c0 2024-01-20T08:31:51.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.967+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab415b530 con 0x7f2ab41539c0 2024-01-20T08:31:51.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.968+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab41e4210 con 0x7f2ab41539c0 2024-01-20T08:31:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.986+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2aac007070 con 0x7f2ab41539c0 2024-01-20T08:31:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.986+0000 7f2aaa7fc640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2a9806af20 0x7f2a9806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.987+0000 7f2ab9967640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2a9806af20 0x7f2a9806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.987+0000 7f2ab9967640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2a9806af20 0x7f2a9806d3e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0006fd0 tx=0x7f2ab0008040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:51.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:51.989+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f2aac0585e0 con 0x7f2ab41539c0 2024-01-20T08:31:52.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.223+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2aac054920 con 0x7f2ab41539c0 2024-01-20T08:31:52.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.396+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ab4074290 con 0x7f2ab41539c0 2024-01-20T08:31:52.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.417+0000 7f2aaa7fc640 1 -- 172.21.15.87:0/183146553 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2aac017020 con 0x7f2ab41539c0 2024-01-20T08:31:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2a9806af20 msgr2=0x7f2a9806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2a9806af20 0x7f2a9806d3e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0006fd0 tx=0x7f2ab0008040 comp rx=0 tx=0).stop 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 msgr2=0x7f2ab4148470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 0x7f2ab4148470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00cab0 tx=0x7f2aac00cf70 comp rx=0 tx=0).stop 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 shutdown_connections 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2a9806af20 0x7f2a9806d3e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab41539c0 0x7f2ab4148470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40c9f50 0x7f2ab41542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 --2- 172.21.15.87:0/183146553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ab40c90a0 0x7f2ab406b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 >> 172.21.15.87:0/183146553 conn(0x7f2ab4070de0 msgr2=0x7f2ab41e5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 shutdown_connections 2024-01-20T08:31:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.419+0000 7f2abbbcb640 1 -- 172.21.15.87:0/183146553 wait complete. 2024-01-20T08:31:52.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-20T08:31:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 Processor -- start 2024-01-20T08:31:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 -- start start 2024-01-20T08:31:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f068c0c2d80 0x7f068c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068c153a40 con 0x7f068c0c97c0 2024-01-20T08:31:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c06d420 con 0x7f068c0c2d80 2024-01-20T08:31:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.533+0000 7f069279e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0c3a60 con 0x7f068c153660 2024-01-20T08:31:52.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.534+0000 7f068bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:52.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.534+0000 7f0690d3b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f068c0c2d80 0x7f068c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:52.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.535+0000 7f068bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c15a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47436/0 (socket says 172.21.15.87:47436) 2024-01-20T08:31:52.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.535+0000 7f068bfff640 1 -- 172.21.15.87:0/1316332252 learned_addr learned my addr 172.21.15.87:0/1316332252 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:52.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.537+0000 7f068bfff640 1 -- 172.21.15.87:0/1316332252 >> v2:172.21.15.116:3300/0 conn(0x7f068c0c2d80 msgr2=0x7f068c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:52.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.537+0000 7f068bfff640 1 --2- 172.21.15.87:0/1316332252 >> v2:172.21.15.116:3300/0 conn(0x7f068c0c2d80 0x7f068c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.537+0000 7f068bfff640 1 -- 172.21.15.87:0/1316332252 >> v1:172.21.15.116:6789/0 conn(0x7f068c0c97c0 legacy=0x7f068c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:52.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.537+0000 7f068bfff640 1 -- 172.21.15.87:0/1316332252 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0c95d0 con 0x7f068c153660 2024-01-20T08:31:52.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.540+0000 7f068bfff640 1 --2- 172.21.15.87:0/1316332252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c15a1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f067c009870 tx=0x7f067c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4ac40cc3f11c8c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f068affd640 1 -- 172.21.15.87:0/1316332252 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c030840 con 0x7f068c153660 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f068affd640 1 -- 172.21.15.87:0/1316332252 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c0309a0 con 0x7f068c153660 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f068affd640 1 -- 172.21.15.87:0/1316332252 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c030d30 con 0x7f068c153660 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 -- 172.21.15.87:0/1316332252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 msgr2=0x7f068c15a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 --2- 172.21.15.87:0/1316332252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c15a1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f067c009870 tx=0x7f067c02b980 comp rx=0 tx=0).stop 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 -- 172.21.15.87:0/1316332252 shutdown_connections 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 --2- 172.21.15.87:0/1316332252 >> v2:172.21.15.116:3300/0 conn(0x7f068c0c2d80 0x7f068c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 --2- 172.21.15.87:0/1316332252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c15a1b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 -- 172.21.15.87:0/1316332252 >> 172.21.15.87:0/1316332252 conn(0x7f068c06a5b0 msgr2=0x7f068c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:52.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 -- 172.21.15.87:0/1316332252 shutdown_connections 2024-01-20T08:31:52.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.545+0000 7f069279e640 1 -- 172.21.15.87:0/1316332252 wait complete. 2024-01-20T08:31:52.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 Processor -- start 2024-01-20T08:31:52.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 -- start start 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068c0c2d80 0x7f068c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f068c0c97c0 0x7f068c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0cafb0 con 0x7f068c0c97c0 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0cb290 con 0x7f068c0c2d80 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.547+0000 7f069279e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0cb570 con 0x7f068c153660 2024-01-20T08:31:52.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.548+0000 7f068b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f068c0c97c0 0x7f068c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:52.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.548+0000 7f068bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068c0c2d80 0x7f068c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:52.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.548+0000 7f068bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068c0c2d80 0x7f068c14b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46722/0 (socket says 172.21.15.87:46722) 2024-01-20T08:31:52.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.548+0000 7f068bfff640 1 -- 172.21.15.87:0/3982598063 learned_addr learned my addr 172.21.15.87:0/3982598063 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:52.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.550+0000 7f0690d3b640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:52.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.552+0000 7f068b7fe640 1 -- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 msgr2=0x7f068c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:52.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.552+0000 7f068b7fe640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.552+0000 7f068b7fe640 1 -- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068c0c2d80 msgr2=0x7f068c14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:52.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.552+0000 7f068b7fe640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068c0c2d80 0x7f068c14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:52.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.552+0000 7f068b7fe640 1 -- 172.21.15.87:0/3982598063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0094d0 con 0x7f068c0c97c0 2024-01-20T08:31:52.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.554+0000 7f068b7fe640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f068c0c97c0 0x7f068c0ccae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f068000c340 tx=0x7f068000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:52.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.557+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06800052d0 con 0x7f068c0c97c0 2024-01-20T08:31:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.557+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680005450 con 0x7f068c0c97c0 2024-01-20T08:31:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.557+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c0cb850 con 0x7f068c0c97c0 2024-01-20T08:31:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.557+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680007260 con 0x7f068c0c97c0 2024-01-20T08:31:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.557+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c14b740 con 0x7f068c0c97c0 2024-01-20T08:31:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.558+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c0cbd70 con 0x7f068c0c97c0 2024-01-20T08:31:52.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.576+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f06800073c0 con 0x7f068c0c97c0 2024-01-20T08:31:52.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.576+0000 7f0688ff9640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f067806af20 0x7f067806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:52.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.577+0000 7f068bfff640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f067806af20 0x7f067806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:52.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.577+0000 7f068bfff640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f067806af20 0x7f067806d3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f067c009870 tx=0x7f067c008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:52.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.580+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0680057c00 con 0x7f068c0c97c0 2024-01-20T08:31:52.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.815+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0680053f40 con 0x7f068c0c97c0 2024-01-20T08:31:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:52.995+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f068c0cbd70 con 0x7f068c0c97c0 2024-01-20T08:31:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.002+0000 7f0688ff9640 1 -- 172.21.15.87:0/3982598063 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f068c0cbd70 con 0x7f068c0c97c0 2024-01-20T08:31:53.005 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-01-20T08:31:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f067806af20 msgr2=0x7f067806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f067806af20 0x7f067806d3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f067c009870 tx=0x7f067c008030 comp rx=0 tx=0).stop 2024-01-20T08:31:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f068c0c97c0 msgr2=0x7f068c0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f068c0c97c0 0x7f068c0ccae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f068000c340 tx=0x7f068000c800 comp rx=0 tx=0).stop 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 shutdown_connections 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f067806af20 0x7f067806d3e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068c153660 0x7f068c0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f068c0c97c0 0x7f068c0ccae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 --2- 172.21.15.87:0/3982598063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068c0c2d80 0x7f068c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 >> 172.21.15.87:0/3982598063 conn(0x7f068c06a5b0 msgr2=0x7f068c151880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 shutdown_connections 2024-01-20T08:31:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.004+0000 7f069279e640 1 -- 172.21.15.87:0/3982598063 wait complete. 2024-01-20T08:31:53.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-20T08:31:53.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.117+0000 7fa47a075640 1 Processor -- start 2024-01-20T08:31:53.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.117+0000 7fa47a075640 1 -- start start 2024-01-20T08:31:53.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.118+0000 7fa47a075640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa4740cae00 0x7fa474072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.118+0000 7fa47a075640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa4740716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.118+0000 7fa47a075640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740c9d60 con 0x7fa474153c60 2024-01-20T08:31:53.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.118+0000 7fa47a075640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4740ca9e0 con 0x7fa4740cae00 2024-01-20T08:31:53.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.118+0000 7fa47a075640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740c3230 con 0x7fa47415d710 2024-01-20T08:31:53.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.119+0000 7fa473fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa4740716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.119+0000 7fa473fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa4740716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47466/0 (socket says 172.21.15.87:47466) 2024-01-20T08:31:53.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.119+0000 7fa473fff640 1 -- 172.21.15.87:0/786921074 learned_addr learned my addr 172.21.15.87:0/786921074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:53.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.119+0000 7fa472ffd640 1 --2- 172.21.15.87:0/786921074 >> v2:172.21.15.116:3300/0 conn(0x7fa4740cae00 0x7fa474072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.121+0000 7fa473fff640 1 -- 172.21.15.87:0/786921074 >> v2:172.21.15.116:3300/0 conn(0x7fa4740cae00 msgr2=0x7fa474072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:53.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.121+0000 7fa473fff640 1 --2- 172.21.15.87:0/786921074 >> v2:172.21.15.116:3300/0 conn(0x7fa4740cae00 0x7fa474072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.121+0000 7fa473fff640 1 -- 172.21.15.87:0/786921074 >> v1:172.21.15.116:6789/0 conn(0x7fa474153c60 legacy=0x7fa47406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.121+0000 7fa473fff640 1 -- 172.21.15.87:0/786921074 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740ca040 con 0x7fa47415d710 2024-01-20T08:31:53.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.123+0000 7fa473fff640 1 --2- 172.21.15.87:0/786921074 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa4740716b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009e50 tx=0x7fa46c02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ca78006ff14a87 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa4727fc640 1 -- 172.21.15.87:0/786921074 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c02cb90 con 0x7fa47415d710 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa4727fc640 1 -- 172.21.15.87:0/786921074 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c02ccf0 con 0x7fa47415d710 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa4727fc640 1 -- 172.21.15.87:0/786921074 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c034610 con 0x7fa47415d710 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 -- 172.21.15.87:0/786921074 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 msgr2=0x7fa4740716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 --2- 172.21.15.87:0/786921074 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa4740716b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009e50 tx=0x7fa46c02bff0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 -- 172.21.15.87:0/786921074 shutdown_connections 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 --2- 172.21.15.87:0/786921074 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa4740716b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 --2- 172.21.15.87:0/786921074 >> v2:172.21.15.116:3300/0 conn(0x7fa4740cae00 0x7fa474072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 -- 172.21.15.87:0/786921074 >> 172.21.15.87:0/786921074 conn(0x7fa4740cb310 msgr2=0x7fa4740cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 -- 172.21.15.87:0/786921074 shutdown_connections 2024-01-20T08:31:53.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.127+0000 7fa47a075640 1 -- 172.21.15.87:0/786921074 wait complete. 2024-01-20T08:31:53.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.128+0000 7fa47a075640 1 Processor -- start 2024-01-20T08:31:53.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 -- start start 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4740cae00 0x7fa47415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474153c60 0x7fa474156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa47414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474159a60 con 0x7fa474153c60 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741efb80 con 0x7fa4740cae00 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.129+0000 7fa47a075640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47415d2b0 con 0x7fa47415d710 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.130+0000 7fa472ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474153c60 0x7fa474156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.130+0000 7fa473fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa47414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.130+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4740cae00 0x7fa47415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.130+0000 7fa473fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa47414dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:47472/0 (socket says 172.21.15.87:47472) 2024-01-20T08:31:53.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.130+0000 7fa473fff640 1 -- 172.21.15.87:0/2374640966 learned_addr learned my addr 172.21.15.87:0/2374640966 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:53.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.133+0000 7fa473fff640 1 -- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4740cae00 msgr2=0x7fa47415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.134+0000 7fa473fff640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4740cae00 0x7fa47415c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.134+0000 7fa473fff640 1 -- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474153c60 msgr2=0x7fa474156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.134+0000 7fa473fff640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474153c60 0x7fa474156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.134+0000 7fa473fff640 1 -- 172.21.15.87:0/2374640966 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c009b30 con 0x7fa47415d710 2024-01-20T08:31:53.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.136+0000 7fa473fff640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa47414dc60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa46c036040 tx=0x7fa46c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:53.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.139+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c009510 con 0x7fa47415d710 2024-01-20T08:31:53.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.139+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c03e450 con 0x7fa47415d710 2024-01-20T08:31:53.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.139+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c03e800 con 0x7fa47415d710 2024-01-20T08:31:53.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.139+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4741eef70 con 0x7fa47415d710 2024-01-20T08:31:53.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.140+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4741ef530 con 0x7fa47415d710 2024-01-20T08:31:53.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.140+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4741582f0 con 0x7fa47415d710 2024-01-20T08:31:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.159+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa46c03e960 con 0x7fa47415d710 2024-01-20T08:31:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.159+0000 7fa45bfff640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa46006af00 0x7fa46006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.160+0000 7fa4737fe640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa46006af00 0x7fa46006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.160+0000 7fa4737fe640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa46006af00 0x7fa46006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa464001780 tx=0x7fa464009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:53.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.382+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fa46c07aee0 con 0x7fa47415d710 2024-01-20T08:31:53.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.393+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa46c077220 con 0x7fa47415d710 2024-01-20T08:31:53.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.563+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa47415d3f0 con 0x7fa47415d710 2024-01-20T08:31:53.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.570+0000 7fa45bfff640 1 -- 172.21.15.87:0/2374640966 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa46c02ce60 con 0x7fa47415d710 2024-01-20T08:31:53.572 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:53.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.571+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa46006af00 msgr2=0x7fa46006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.571+0000 7fa47a075640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa46006af00 0x7fa46006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa464001780 tx=0x7fa464009040 comp rx=0 tx=0).stop 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.571+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 msgr2=0x7fa47414dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa47414dc60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa46c036040 tx=0x7fa46c004a40 comp rx=0 tx=0).stop 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 shutdown_connections 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa46006af00 0x7fa46006d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa47415d710 0x7fa47414dc60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474153c60 0x7fa474156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 --2- 172.21.15.87:0/2374640966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4740cae00 0x7fa47415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 >> 172.21.15.87:0/2374640966 conn(0x7fa4740cb310 msgr2=0x7fa4740cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:53.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 shutdown_connections 2024-01-20T08:31:53.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.572+0000 7fa47a075640 1 -- 172.21.15.87:0/2374640966 wait complete. 2024-01-20T08:31:53.586 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.683+0000 7efdaac61640 1 Processor -- start 2024-01-20T08:31:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.683+0000 7efdaac61640 1 -- start start 2024-01-20T08:31:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.684+0000 7efdaac61640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efda40cd020 0x7efda4158030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.684+0000 7efdaac61640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efda40c7c50 0x7efda4151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.684+0000 7efdaac61640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda4158570 con 0x7efda40cd400 2024-01-20T08:31:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.684+0000 7efdaac61640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40c8620 con 0x7efda40cd020 2024-01-20T08:31:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.684+0000 7efdaac61640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40b02e0 con 0x7efda40c7c50 2024-01-20T08:31:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.684+0000 7efda89fd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efda40cd020 0x7efda4158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.685+0000 7efda91fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efda40c7c50 0x7efda4151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.685+0000 7efda89fd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efda40cd020 0x7efda4158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45860/0 (socket says 172.21.15.87:45860) 2024-01-20T08:31:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.685+0000 7efda89fd640 1 -- 172.21.15.87:0/2414196003 learned_addr learned my addr 172.21.15.87:0/2414196003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:53.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.687+0000 7efda89fd640 1 -- 172.21.15.87:0/2414196003 >> v2:172.21.15.116:3300/0 conn(0x7efda40c7c50 msgr2=0x7efda4151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:53.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.687+0000 7efda89fd640 1 --2- 172.21.15.87:0/2414196003 >> v2:172.21.15.116:3300/0 conn(0x7efda40c7c50 0x7efda4151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.687+0000 7efda89fd640 1 -- 172.21.15.87:0/2414196003 >> v1:172.21.15.116:6789/0 conn(0x7efda40cd400 legacy=0x7efda40c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.687+0000 7efda89fd640 1 -- 172.21.15.87:0/2414196003 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40c9200 con 0x7efda40cd020 2024-01-20T08:31:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.689+0000 7efda89fd640 1 --2- 172.21.15.87:0/2414196003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda4158030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efda40cd810 tx=0x7efd9802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b076a9dfbb3f8a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efda37fe640 1 -- 172.21.15.87:0/2414196003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9802f850 con 0x7efda40cd020 2024-01-20T08:31:53.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efda37fe640 1 -- 172.21.15.87:0/2414196003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9802f9b0 con 0x7efda40cd020 2024-01-20T08:31:53.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efda37fe640 1 -- 172.21.15.87:0/2414196003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9802fd40 con 0x7efda40cd020 2024-01-20T08:31:53.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 -- 172.21.15.87:0/2414196003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 msgr2=0x7efda4158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 --2- 172.21.15.87:0/2414196003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda4158030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efda40cd810 tx=0x7efd9802ba30 comp rx=0 tx=0).stop 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 -- 172.21.15.87:0/2414196003 shutdown_connections 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 --2- 172.21.15.87:0/2414196003 >> v2:172.21.15.116:3300/0 conn(0x7efda40c7c50 0x7efda4151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 --2- 172.21.15.87:0/2414196003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda4158030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 -- 172.21.15.87:0/2414196003 >> 172.21.15.87:0/2414196003 conn(0x7efda406a5b0 msgr2=0x7efda406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 -- 172.21.15.87:0/2414196003 shutdown_connections 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.693+0000 7efdaac61640 1 -- 172.21.15.87:0/2414196003 wait complete. 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 Processor -- start 2024-01-20T08:31:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 -- start start 2024-01-20T08:31:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efda40c7c50 0x7efda40c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda40c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efda40cd400 0x7efda41f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40c4de0 con 0x7efda40cd020 2024-01-20T08:31:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda4157ba0 con 0x7efda40cd400 2024-01-20T08:31:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.695+0000 7efdaac61640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40c5b20 con 0x7efda40c7c50 2024-01-20T08:31:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.696+0000 7efda3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda40c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.696+0000 7efda91fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efda40cd400 0x7efda41f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.696+0000 7efda89fd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efda40c7c50 0x7efda40c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.696+0000 7efda3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda40c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45866/0 (socket says 172.21.15.87:45866) 2024-01-20T08:31:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.696+0000 7efda3fff640 1 -- 172.21.15.87:0/2480653864 learned_addr learned my addr 172.21.15.87:0/2480653864 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.699+0000 7efda3fff640 1 -- 172.21.15.87:0/2480653864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efda40c7c50 msgr2=0x7efda40c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.699+0000 7efda3fff640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efda40c7c50 0x7efda40c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.699+0000 7efda3fff640 1 -- 172.21.15.87:0/2480653864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efda40cd400 msgr2=0x7efda41f08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.699+0000 7efda3fff640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efda40cd400 0x7efda41f08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.699+0000 7efda3fff640 1 -- 172.21.15.87:0/2480653864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980094d0 con 0x7efda40cd020 2024-01-20T08:31:53.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.702+0000 7efda3fff640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda40c64f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c400 tx=0x7efd9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:53.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.705+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900052d0 con 0x7efda40cd020 2024-01-20T08:31:53.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.705+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd90005430 con 0x7efda40cd020 2024-01-20T08:31:53.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.705+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda41578c0 con 0x7efda40cd020 2024-01-20T08:31:53.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.705+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90003830 con 0x7efda40cd020 2024-01-20T08:31:53.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.705+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda40c54d0 con 0x7efda40cd020 2024-01-20T08:31:53.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.706+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda4074290 con 0x7efda40cd020 2024-01-20T08:31:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.726+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efd90003990 con 0x7efda40cd020 2024-01-20T08:31:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.726+0000 7efda17fa640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efd8806af20 0x7efd8806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.726+0000 7efda89fd640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efd8806af20 0x7efd8806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:53.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.727+0000 7efda89fd640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efd8806af20 0x7efd8806d3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd98036000 tx=0x7efd98008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:53.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.958+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7efd90057a60 con 0x7efda40cd020 2024-01-20T08:31:53.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:53.958+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efd90053da0 con 0x7efda40cd020 2024-01-20T08:31:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.169+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda4158570 con 0x7efda40cd020 2024-01-20T08:31:54.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.190+0000 7efda17fa640 1 -- 172.21.15.87:0/2480653864 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6584 (secure 0 0 0) 0x7efda4158570 con 0x7efda40cd020 2024-01-20T08:31:54.194 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:54.194 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:45.461258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4290,"mds_2":4287,"mds_3":4284,"mds_4":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":4,"incarnation":17,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":12,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4287":{"gid":4287,"name":"c","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4290":{"gid":4290,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-20T08:31:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 18 2024-01-20T08:31:54.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efd8806af20 msgr2=0x7efd8806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efd8806af20 0x7efd8806d3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd98036000 tx=0x7efd98008030 comp rx=0 tx=0).stop 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 msgr2=0x7efda40c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda40c64f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c400 tx=0x7efd9000c8c0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 shutdown_connections 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7efd8806af20 0x7efd8806d3e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efda40cd400 0x7efda41f08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda40cd020 0x7efda40c64f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 --2- 172.21.15.87:0/2480653864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efda40c7c50 0x7efda40c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 >> 172.21.15.87:0/2480653864 conn(0x7efda406a5b0 msgr2=0x7efda41511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 shutdown_connections 2024-01-20T08:31:54.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.192+0000 7efdaac61640 1 -- 172.21.15.87:0/2480653864 wait complete. 2024-01-20T08:31:54.206 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 Processor -- start 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 -- start start 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56d80ca090 0x7f56d80cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56d815d6f0 0x7f56d80c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d8153eb0 con 0x7f56d8153ad0 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d815a560 con 0x7f56d80ca090 2024-01-20T08:31:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.305+0000 7f56defc9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d815dc80 con 0x7f56d815d6f0 2024-01-20T08:31:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.306+0000 7f56dd566640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56d815d6f0 0x7f56d80c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.306+0000 7f56dd566640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56d815d6f0 0x7f56d80c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46744/0 (socket says 172.21.15.87:46744) 2024-01-20T08:31:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.306+0000 7f56dd566640 1 -- 172.21.15.87:0/632466663 learned_addr learned my addr 172.21.15.87:0/632466663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:54.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.309+0000 7f56dd566640 1 -- 172.21.15.87:0/632466663 >> v2:172.21.15.87:3300/0 conn(0x7f56d80ca090 msgr2=0x7f56d80cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.309+0000 7f56dd566640 1 --2- 172.21.15.87:0/632466663 >> v2:172.21.15.87:3300/0 conn(0x7f56d80ca090 0x7f56d80cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.309+0000 7f56dd566640 1 -- 172.21.15.87:0/632466663 >> v1:172.21.15.87:6789/0 conn(0x7f56d8153ad0 legacy=0x7f56d80ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.309+0000 7f56dd566640 1 -- 172.21.15.87:0/632466663 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80cb760 con 0x7f56d815d6f0 2024-01-20T08:31:54.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.310+0000 7f56dd566640 1 --2- 172.21.15.87:0/632466663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d815d6f0 0x7f56d80c3e60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56d80ca8a0 tx=0x7f56c002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da2399a34602d37 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.314+0000 7f56d77fe640 1 -- 172.21.15.87:0/632466663 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031870 con 0x7f56d815d6f0 2024-01-20T08:31:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.314+0000 7f56d77fe640 1 -- 172.21.15.87:0/632466663 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c00319d0 con 0x7f56d815d6f0 2024-01-20T08:31:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.314+0000 7f56d77fe640 1 -- 172.21.15.87:0/632466663 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031d60 con 0x7f56d815d6f0 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 -- 172.21.15.87:0/632466663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d815d6f0 msgr2=0x7f56d80c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 --2- 172.21.15.87:0/632466663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d815d6f0 0x7f56d80c3e60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56d80ca8a0 tx=0x7f56c002b910 comp rx=0 tx=0).stop 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 -- 172.21.15.87:0/632466663 shutdown_connections 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 --2- 172.21.15.87:0/632466663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d815d6f0 0x7f56d80c3e60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 --2- 172.21.15.87:0/632466663 >> v2:172.21.15.87:3300/0 conn(0x7f56d80ca090 0x7f56d80cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 -- 172.21.15.87:0/632466663 >> 172.21.15.87:0/632466663 conn(0x7f56d808af90 msgr2=0x7f56d808b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 -- 172.21.15.87:0/632466663 shutdown_connections 2024-01-20T08:31:54.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.315+0000 7f56defc9640 1 -- 172.21.15.87:0/632466663 wait complete. 2024-01-20T08:31:54.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.316+0000 7f56defc9640 1 Processor -- start 2024-01-20T08:31:54.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.316+0000 7f56defc9640 1 -- start start 2024-01-20T08:31:54.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56defc9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56d80ca090 0x7f56d80c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56defc9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 0x7f56d814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56defc9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d815d6f0 0x7f56d80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56defc9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d814f130 con 0x7f56d815d6f0 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56defc9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d81f2d80 con 0x7f56d8153ad0 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56defc9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80b4f10 con 0x7f56d80ca090 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 0x7f56d814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.317+0000 7f56dcd65640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56d80ca090 0x7f56d80c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.318+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 0x7f56d814f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46748/0 (socket says 172.21.15.87:46748) 2024-01-20T08:31:54.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.318+0000 7f56d7fff640 1 -- 172.21.15.87:0/2352048636 learned_addr learned my addr 172.21.15.87:0/2352048636 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:54.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.321+0000 7f56d7fff640 1 -- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56d80ca090 msgr2=0x7f56d80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.321+0000 7f56d7fff640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56d80ca090 0x7f56d80c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.321+0000 7f56d7fff640 1 -- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d815d6f0 msgr2=0x7f56d80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.321+0000 7f56d7fff640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d815d6f0 0x7f56d80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.321+0000 7f56d7fff640 1 -- 172.21.15.87:0/2352048636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00094d0 con 0x7f56d8153ad0 2024-01-20T08:31:54.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.324+0000 7f56d7fff640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 0x7f56d814f5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56c800ba50 tx=0x7f56c800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.328+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c800cbe0 con 0x7f56d8153ad0 2024-01-20T08:31:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.328+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c800cd60 con 0x7f56d8153ad0 2024-01-20T08:31:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.328+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8014680 con 0x7f56d8153ad0 2024-01-20T08:31:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.328+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d814fb30 con 0x7f56d8153ad0 2024-01-20T08:31:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.328+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d8147490 con 0x7f56d8153ad0 2024-01-20T08:31:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.329+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d8159ba0 con 0x7f56d8153ad0 2024-01-20T08:31:54.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.347+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f56c801d3b0 con 0x7f56d8153ad0 2024-01-20T08:31:54.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.347+0000 7f56d57fa640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56b806af20 0x7f56b806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.347+0000 7f56dcd65640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56b806af20 0x7f56b806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.348+0000 7f56dcd65640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56b806af20 0x7f56b806d3e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56d00017f0 tx=0x7f56d0009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:54.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.350+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f56c8058460 con 0x7f56d8153ad0 2024-01-20T08:31:54.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.638+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f56c8087e60 con 0x7f56d8153ad0 2024-01-20T08:31:54.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.810+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56d81f2ec0 con 0x7f56d8153ad0 2024-01-20T08:31:54.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.848+0000 7f56d57fa640 1 -- 172.21.15.87:0/2352048636 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7f56d81f2ec0 con 0x7f56d8153ad0 2024-01-20T08:31:54.852 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:54.852 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:54.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56b806af20 msgr2=0x7f56b806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56b806af20 0x7f56b806d3e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56d00017f0 tx=0x7f56d0009040 comp rx=0 tx=0).stop 2024-01-20T08:31:54.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 msgr2=0x7f56d814f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 0x7f56d814f5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56c800ba50 tx=0x7f56c800bf10 comp rx=0 tx=0).stop 2024-01-20T08:31:54.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 shutdown_connections 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56b806af20 0x7f56b806d3e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d815d6f0 0x7f56d80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d8153ad0 0x7f56d814f5f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 --2- 172.21.15.87:0/2352048636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56d80ca090 0x7f56d80c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 >> 172.21.15.87:0/2352048636 conn(0x7f56d808af90 msgr2=0x7f56d8151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 shutdown_connections 2024-01-20T08:31:54.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.851+0000 7f56defc9640 1 -- 172.21.15.87:0/2352048636 wait complete. 2024-01-20T08:31:54.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-20T08:31:06.500533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-20T08:31:08.175669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-20T08:31:13.903374+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-20T08:31:54.865 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:31:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.965+0000 7f32350e5640 1 Processor -- start 2024-01-20T08:31:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.965+0000 7f32350e5640 1 -- start start 2024-01-20T08:31:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.966+0000 7f32350e5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32300c6760 0x7f32300c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.966+0000 7f32350e5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32300c2b20 0x7f32300c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.966+0000 7f32350e5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323006ea70 con 0x7f32301502e0 2024-01-20T08:31:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.966+0000 7f32350e5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32300c2f50 con 0x7f32300c2b20 2024-01-20T08:31:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.966+0000 7f32350e5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323006ed10 con 0x7f32300c6760 2024-01-20T08:31:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.966+0000 7f322f59e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32300c2b20 0x7f32300c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.967+0000 7f322f59e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32300c2b20 0x7f32300c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45888/0 (socket says 172.21.15.87:45888) 2024-01-20T08:31:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.967+0000 7f322f59e640 1 -- 172.21.15.87:0/3353952029 learned_addr learned my addr 172.21.15.87:0/3353952029 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.967+0000 7f322659c640 1 --2- 172.21.15.87:0/3353952029 >> v2:172.21.15.116:3300/0 conn(0x7f32300c6760 0x7f32300c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.969+0000 7f322f59e640 1 -- 172.21.15.87:0/3353952029 >> v2:172.21.15.116:3300/0 conn(0x7f32300c6760 msgr2=0x7f32300c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:54.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.969+0000 7f322f59e640 1 --2- 172.21.15.87:0/3353952029 >> v2:172.21.15.116:3300/0 conn(0x7f32300c6760 0x7f32300c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.969+0000 7f322f59e640 1 -- 172.21.15.87:0/3353952029 >> v1:172.21.15.116:6789/0 conn(0x7f32301502e0 legacy=0x7f32300c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.969+0000 7f322f59e640 1 -- 172.21.15.87:0/3353952029 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323006dd90 con 0x7f32300c2b20 2024-01-20T08:31:54.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.971+0000 7f322f59e640 1 --2- 172.21.15.87:0/3353952029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32300c2b20 0x7f32300c8080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32300c3130 tx=0x7f322802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319f4ffaddfd530 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.978+0000 7f322e59c640 1 -- 172.21.15.87:0/3353952029 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322802cbb0 con 0x7f32300c2b20 2024-01-20T08:31:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.980+0000 7f322e59c640 1 -- 172.21.15.87:0/3353952029 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322802cd10 con 0x7f32300c2b20 2024-01-20T08:31:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.980+0000 7f322e59c640 1 -- 172.21.15.87:0/3353952029 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228034610 con 0x7f32300c2b20 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 -- 172.21.15.87:0/3353952029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32300c2b20 msgr2=0x7f32300c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 --2- 172.21.15.87:0/3353952029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32300c2b20 0x7f32300c8080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32300c3130 tx=0x7f322802c010 comp rx=0 tx=0).stop 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 -- 172.21.15.87:0/3353952029 shutdown_connections 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 --2- 172.21.15.87:0/3353952029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32300c2b20 0x7f32300c8080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 --2- 172.21.15.87:0/3353952029 >> v2:172.21.15.116:3300/0 conn(0x7f32300c6760 0x7f32300c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 -- 172.21.15.87:0/3353952029 >> 172.21.15.87:0/3353952029 conn(0x7f32300723c0 msgr2=0x7f323006fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:54.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 -- 172.21.15.87:0/3353952029 shutdown_connections 2024-01-20T08:31:54.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.981+0000 7f32350e5640 1 -- 172.21.15.87:0/3353952029 wait complete. 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.983+0000 7f32350e5640 1 Processor -- start 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.983+0000 7f32350e5640 1 -- start start 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f32350e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 0x7f3230141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f32350e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32300c6760 0x7f32301ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f32350e5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32301502e0 0x7f32301ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f32350e5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323015a860 con 0x7f32301502e0 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f32350e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301f32d0 con 0x7f32300c2b20 2024-01-20T08:31:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f32350e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323006b720 con 0x7f32300c6760 2024-01-20T08:31:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f322ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 0x7f3230141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.984+0000 7f322659c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32300c6760 0x7f32301ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.985+0000 7f322ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 0x7f3230141010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:46764/0 (socket says 172.21.15.87:46764) 2024-01-20T08:31:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.985+0000 7f322ed9d640 1 -- 172.21.15.87:0/2502141861 learned_addr learned my addr 172.21.15.87:0/2502141861 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.986+0000 7f322f59e640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32301502e0 0x7f32301ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.988+0000 7f322ed9d640 1 -- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32300c6760 msgr2=0x7f32301ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.988+0000 7f322ed9d640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32300c6760 0x7f32301ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.988+0000 7f322ed9d640 1 -- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32301502e0 msgr2=0x7f32301ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.988+0000 7f322ed9d640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32301502e0 0x7f32301ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.988+0000 7f322ed9d640 1 -- 172.21.15.87:0/2502141861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228009b30 con 0x7f32300c2b20 2024-01-20T08:31:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.988+0000 7f322659c640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32300c6760 0x7f32301ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.991+0000 7f322ed9d640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 0x7f3230141010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f322000c3c0 tx=0x7f322000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.993+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200052d0 con 0x7f32300c2b20 2024-01-20T08:31:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.993+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323006ba00 con 0x7f32300c2b20 2024-01-20T08:31:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.993+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323006bfc0 con 0x7f32300c2b20 2024-01-20T08:31:54.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:54.994+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230074290 con 0x7f32300c2b20 2024-01-20T08:31:55.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.240+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220005430 con 0x7f32300c2b20 2024-01-20T08:31:55.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.240+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220007250 con 0x7f32300c2b20 2024-01-20T08:31:55.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.240+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3220007470 con 0x7f32300c2b20 2024-01-20T08:31:55.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.241+0000 7f3227fff640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321006aed0 0x7f321006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.241+0000 7f322659c640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321006aed0 0x7f321006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.241+0000 7f322659c640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321006aed0 0x7f321006d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3230142e20 tx=0x7f3218009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:55.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.245+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f3220057a80 con 0x7f32300c2b20 2024-01-20T08:31:55.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.256+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3220053dc0 con 0x7f32300c2b20 2024-01-20T08:31:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.428+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f323006fd20 con 0x7f32300c2b20 2024-01-20T08:31:55.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.435+0000 7f3227fff640 1 -- 172.21.15.87:0/2502141861 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3220020020 con 0x7f32300c2b20 2024-01-20T08:31:55.438 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:31:55.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321006aed0 msgr2=0x7f321006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321006aed0 0x7f321006d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3230142e20 tx=0x7f3218009190 comp rx=0 tx=0).stop 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 msgr2=0x7f3230141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 0x7f3230141010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f322000c3c0 tx=0x7f322000c880 comp rx=0 tx=0).stop 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 shutdown_connections 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f321006aed0 0x7f321006d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32301502e0 0x7f32301ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32300c6760 0x7f32301ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 --2- 172.21.15.87:0/2502141861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32300c2b20 0x7f3230141010 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 >> 172.21.15.87:0/2502141861 conn(0x7f32300723c0 msgr2=0x7f32300c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 shutdown_connections 2024-01-20T08:31:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.438+0000 7f32350e5640 1 -- 172.21.15.87:0/2502141861 wait complete. 2024-01-20T08:31:55.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-20T08:31:55.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.552+0000 7f1c5ef89640 1 Processor -- start 2024-01-20T08:31:55.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.552+0000 7f1c5ef89640 1 -- start start 2024-01-20T08:31:55.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.553+0000 7f1c5ef89640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c5815a1b0 0x7f1c580c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.553+0000 7f1c5ef89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c580c2d80 0x7f1c580ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.553+0000 7f1c5ef89640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c58153a40 con 0x7f1c58153660 2024-01-20T08:31:55.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.553+0000 7f1c5ef89640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5806d420 con 0x7f1c580c2d80 2024-01-20T08:31:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.553+0000 7f1c5ef89640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c580c3a60 con 0x7f1c5815a1b0 2024-01-20T08:31:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.554+0000 7f1c5d526640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c580c2d80 0x7f1c580ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.554+0000 7f1c57fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c5815a1b0 0x7f1c580c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.554+0000 7f1c57fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c5815a1b0 0x7f1c580c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41884/0 (socket says 172.21.15.87:41884) 2024-01-20T08:31:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.554+0000 7f1c57fff640 1 -- 172.21.15.87:0/2696379559 learned_addr learned my addr 172.21.15.87:0/2696379559 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.556+0000 7f1c57fff640 1 -- 172.21.15.87:0/2696379559 >> v2:172.21.15.87:3300/0 conn(0x7f1c580c2d80 msgr2=0x7f1c580ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.556+0000 7f1c57fff640 1 --2- 172.21.15.87:0/2696379559 >> v2:172.21.15.87:3300/0 conn(0x7f1c580c2d80 0x7f1c580ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.556+0000 7f1c57fff640 1 -- 172.21.15.87:0/2696379559 >> v1:172.21.15.87:6789/0 conn(0x7f1c58153660 legacy=0x7f1c580c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.556+0000 7f1c57fff640 1 -- 172.21.15.87:0/2696379559 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580c95d0 con 0x7f1c5815a1b0 2024-01-20T08:31:55.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.558+0000 7f1c57fff640 1 --2- 172.21.15.87:0/2696379559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c5815a1b0 0x7f1c580c97c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c580cd8f0 tx=0x7f1c4c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb76135cbc0d4f6d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:55.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c577fe640 1 -- 172.21.15.87:0/2696379559 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c030860 con 0x7f1c5815a1b0 2024-01-20T08:31:55.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c577fe640 1 -- 172.21.15.87:0/2696379559 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c0309c0 con 0x7f1c5815a1b0 2024-01-20T08:31:55.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c577fe640 1 -- 172.21.15.87:0/2696379559 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c030d50 con 0x7f1c5815a1b0 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2696379559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c5815a1b0 msgr2=0x7f1c580c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2696379559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c5815a1b0 0x7f1c580c97c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c580cd8f0 tx=0x7f1c4c02b9a0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2696379559 shutdown_connections 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2696379559 >> v2:172.21.15.87:3300/0 conn(0x7f1c580c2d80 0x7f1c580ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2696379559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c5815a1b0 0x7f1c580c97c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2696379559 >> 172.21.15.87:0/2696379559 conn(0x7f1c5806a5b0 msgr2=0x7f1c5806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2696379559 shutdown_connections 2024-01-20T08:31:55.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.562+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2696379559 wait complete. 2024-01-20T08:31:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.563+0000 7f1c5ef89640 1 Processor -- start 2024-01-20T08:31:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 -- start start 2024-01-20T08:31:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c580c2d80 0x7f1c5815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 0x7f1c581f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1c5815a1b0 0x7f1c580cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5815b8b0 con 0x7f1c580c2d80 2024-01-20T08:31:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581f3190 con 0x7f1c58153660 2024-01-20T08:31:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.564+0000 7f1c5ef89640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5814e580 con 0x7f1c5815a1b0 2024-01-20T08:31:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.565+0000 7f1c5cd25640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c580c2d80 0x7f1c5815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.565+0000 7f1c57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 0x7f1c581f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.565+0000 7f1c5d526640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1c5815a1b0 0x7f1c580cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.565+0000 7f1c57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 0x7f1c581f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41888/0 (socket says 172.21.15.87:41888) 2024-01-20T08:31:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.565+0000 7f1c57fff640 1 -- 172.21.15.87:0/2661747266 learned_addr learned my addr 172.21.15.87:0/2661747266 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:55.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.568+0000 7f1c5cd25640 1 -- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1c5815a1b0 msgr2=0x7f1c580cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:55.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.568+0000 7f1c5cd25640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1c5815a1b0 0x7f1c580cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.568+0000 7f1c5cd25640 1 -- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 msgr2=0x7f1c581f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:55.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.568+0000 7f1c5cd25640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 0x7f1c581f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:55.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.568+0000 7f1c5cd25640 1 -- 172.21.15.87:0/2661747266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0094d0 con 0x7f1c580c2d80 2024-01-20T08:31:55.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.568+0000 7f1c57fff640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 0x7f1c581f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:55.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.571+0000 7f1c5cd25640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c580c2d80 0x7f1c5815b370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800c340 tx=0x7f1c4800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:55.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.574+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480052d0 con 0x7f1c580c2d80 2024-01-20T08:31:55.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.574+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c48005450 con 0x7f1c580c2d80 2024-01-20T08:31:55.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.574+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c581f2eb0 con 0x7f1c580c2d80 2024-01-20T08:31:55.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.574+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48007260 con 0x7f1c580c2d80 2024-01-20T08:31:55.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.574+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c580acee0 con 0x7f1c580c2d80 2024-01-20T08:31:55.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.575+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c58074290 con 0x7f1c580c2d80 2024-01-20T08:31:55.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.594+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1c480073c0 con 0x7f1c580c2d80 2024-01-20T08:31:55.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.595+0000 7f1c557fa640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1c4406af50 0x7f1c4406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:55.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.595+0000 7f1c57fff640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1c4406af50 0x7f1c4406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:55.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.595+0000 7f1c57fff640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1c4406af50 0x7f1c4406d410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c580b0ae0 tx=0x7f1c4c008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.827+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f1c48057da0 con 0x7f1c580c2d80 2024-01-20T08:31:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.827+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1c480540e0 con 0x7f1c580c2d80 2024-01-20T08:31:55.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:55.996+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c580c86b0 con 0x7f1c580c2d80 2024-01-20T08:31:56.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.001+0000 7f1c557fa640 1 -- 172.21.15.87:0/2661747266 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c480200a0 con 0x7f1c580c2d80 2024-01-20T08:31:56.004 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1c4406af50 msgr2=0x7f1c4406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1c4406af50 0x7f1c4406d410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c580b0ae0 tx=0x7f1c4c008030 comp rx=0 tx=0).stop 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c580c2d80 msgr2=0x7f1c5815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c580c2d80 0x7f1c5815b370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800c340 tx=0x7f1c4800c800 comp rx=0 tx=0).stop 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 shutdown_connections 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f1c4406af50 0x7f1c4406d410 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1c5815a1b0 0x7f1c580cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c58153660 0x7f1c581f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 --2- 172.21.15.87:0/2661747266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c580c2d80 0x7f1c5815b370 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 >> 172.21.15.87:0/2661747266 conn(0x7f1c5806a5b0 msgr2=0x7f1c58070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:56.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 shutdown_connections 2024-01-20T08:31:56.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.004+0000 7f1c5ef89640 1 -- 172.21.15.87:0/2661747266 wait complete. 2024-01-20T08:31:56.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-20T08:31:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.116+0000 7f6daf7a9640 1 Processor -- start 2024-01-20T08:31:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.117+0000 7f6daf7a9640 1 -- start start 2024-01-20T08:31:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.117+0000 7f6daf7a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6da814cc40 0x7f6da8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.117+0000 7f6daf7a9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da8156720 con 0x7f6da815ec50 2024-01-20T08:31:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.117+0000 7f6daf7a9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80c19c0 con 0x7f6da80bfe70 2024-01-20T08:31:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.117+0000 7f6daf7a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da8072b30 con 0x7f6da814cc40 2024-01-20T08:31:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.118+0000 7f6dad545640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6da814cc40 0x7f6da8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.118+0000 7f6dacd44640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6da80bfe70 0x7f6da80c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35562/0 (socket says 172.21.15.87:35562) 2024-01-20T08:31:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.118+0000 7f6dacd44640 1 -- 172.21.15.87:0/2423217156 learned_addr learned my addr 172.21.15.87:0/2423217156 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:56.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.120+0000 7f6d9ffff640 1 -- 172.21.15.87:0/2423217156 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135955740 0 0) 0x7f6da80c19c0 con 0x7f6da80bfe70 2024-01-20T08:31:56.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.120+0000 7f6d9ffff640 1 -- 172.21.15.87:0/2423217156 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d94003520 con 0x7f6da80bfe70 2024-01-20T08:31:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.123+0000 7f6dad545640 1 -- 172.21.15.87:0/2423217156 >> v1:172.21.15.116:6789/0 conn(0x7f6da80bfe70 legacy=0x7f6da80c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.123+0000 7f6dad545640 1 -- 172.21.15.87:0/2423217156 >> v1:172.21.15.87:6789/0 conn(0x7f6da815ec50 legacy=0x7f6da8161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.123+0000 7f6dad545640 1 -- 172.21.15.87:0/2423217156 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da81563f0 con 0x7f6da814cc40 2024-01-20T08:31:56.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.126+0000 7f6dad545640 1 --2- 172.21.15.87:0/2423217156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da814cc40 0x7f6da8156bb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d980097a0 tx=0x7f6d9802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95a1ed8d1d9131b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:56.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6d9ffff640 1 -- 172.21.15.87:0/2423217156 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9802f810 con 0x7f6da814cc40 2024-01-20T08:31:56.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6d9ffff640 1 -- 172.21.15.87:0/2423217156 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9802f970 con 0x7f6da814cc40 2024-01-20T08:31:56.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6d9ffff640 1 -- 172.21.15.87:0/2423217156 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9802fd00 con 0x7f6da814cc40 2024-01-20T08:31:56.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 -- 172.21.15.87:0/2423217156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da814cc40 msgr2=0x7f6da8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/2423217156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da814cc40 0x7f6da8156bb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d980097a0 tx=0x7f6d9802b9f0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 -- 172.21.15.87:0/2423217156 shutdown_connections 2024-01-20T08:31:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/2423217156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da814cc40 0x7f6da8156bb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 -- 172.21.15.87:0/2423217156 >> 172.21.15.87:0/2423217156 conn(0x7f6da806a9a0 msgr2=0x7f6da806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 -- 172.21.15.87:0/2423217156 shutdown_connections 2024-01-20T08:31:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.130+0000 7f6daf7a9640 1 -- 172.21.15.87:0/2423217156 wait complete. 2024-01-20T08:31:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.132+0000 7f6daf7a9640 1 Processor -- start 2024-01-20T08:31:56.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.132+0000 7f6daf7a9640 1 -- start start 2024-01-20T08:31:56.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.132+0000 7f6daf7a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 0x7f6da80c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.132+0000 7f6daf7a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6da814cc40 0x7f6da80c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6daf7a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6da815ec50 0x7f6da8145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6daf7a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da814ec50 con 0x7f6da80bfe70 2024-01-20T08:31:56.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6daf7a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da814ff40 con 0x7f6da814cc40 2024-01-20T08:31:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6daf7a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8150220 con 0x7f6da815ec50 2024-01-20T08:31:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6dad545640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 0x7f6da80c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6dacd44640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6da814cc40 0x7f6da80c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6dadd46640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6da815ec50 0x7f6da8145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6dad545640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 0x7f6da80c35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34170/0 (socket says 172.21.15.87:34170) 2024-01-20T08:31:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.133+0000 7f6dad545640 1 -- 172.21.15.87:0/263115802 learned_addr learned my addr 172.21.15.87:0/263115802 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.137+0000 7f6dad545640 1 -- 172.21.15.87:0/263115802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6da815ec50 msgr2=0x7f6da8145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.137+0000 7f6dad545640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6da815ec50 0x7f6da8145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.137+0000 7f6dad545640 1 -- 172.21.15.87:0/263115802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6da814cc40 msgr2=0x7f6da80c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.137+0000 7f6dad545640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6da814cc40 0x7f6da80c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.137+0000 7f6dad545640 1 -- 172.21.15.87:0/263115802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98009400 con 0x7f6da80bfe70 2024-01-20T08:31:56.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.139+0000 7f6dad545640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 0x7f6da80c35d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d9802c000 tx=0x7f6d98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:56.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.142+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9803c040 con 0x7f6da80bfe70 2024-01-20T08:31:56.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.142+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d980378f0 con 0x7f6da80bfe70 2024-01-20T08:31:56.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.142+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98037ca0 con 0x7f6da80bfe70 2024-01-20T08:31:56.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.142+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da80c3d40 con 0x7f6da80bfe70 2024-01-20T08:31:56.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.143+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da80c5000 con 0x7f6da80bfe70 2024-01-20T08:31:56.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.144+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da80c4260 con 0x7f6da80bfe70 2024-01-20T08:31:56.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.163+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6d980423f0 con 0x7f6da80bfe70 2024-01-20T08:31:56.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.163+0000 7f6d9dffb640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6d8c06af50 0x7f6d8c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.163+0000 7f6dacd44640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6d8c06af50 0x7f6d8c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.164+0000 7f6dacd44640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6d8c06af50 0x7f6d8c06d410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6da80c4970 tx=0x7f6da4007d30 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:56.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.414+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f6d9807a0c0 con 0x7f6da80bfe70 2024-01-20T08:31:56.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.414+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6d98076400 con 0x7f6da80bfe70 2024-01-20T08:31:56.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.599+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6da80c4260 con 0x7f6da80bfe70 2024-01-20T08:31:56.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.604+0000 7f6d9dffb640 1 -- 172.21.15.87:0/263115802 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d98079a20 con 0x7f6da80bfe70 2024-01-20T08:31:56.607 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-01-20T08:31:56.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6d8c06af50 msgr2=0x7f6d8c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6d8c06af50 0x7f6d8c06d410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6da80c4970 tx=0x7f6da4007d30 comp rx=0 tx=0).stop 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 msgr2=0x7f6da80c35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 0x7f6da80c35d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d9802c000 tx=0x7f6d98004730 comp rx=0 tx=0).stop 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 shutdown_connections 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6d8c06af50 0x7f6d8c06d410 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6da815ec50 0x7f6da8145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6da814cc40 0x7f6da80c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 --2- 172.21.15.87:0/263115802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da80bfe70 0x7f6da80c35d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 >> 172.21.15.87:0/263115802 conn(0x7f6da806a9a0 msgr2=0x7f6da8155490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 shutdown_connections 2024-01-20T08:31:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.606+0000 7f6daf7a9640 1 -- 172.21.15.87:0/263115802 wait complete. 2024-01-20T08:31:56.620 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-20T08:31:56.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.720+0000 7fdd18905640 1 Processor -- start 2024-01-20T08:31:56.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.720+0000 7fdd18905640 1 -- start start 2024-01-20T08:31:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.721+0000 7fdd18905640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdd14151780 0x7fdd1406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.721+0000 7fdd18905640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 0x7fdd1406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.721+0000 7fdd18905640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd14157530 con 0x7fdd141513a0 2024-01-20T08:31:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.721+0000 7fdd18905640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd14158470 con 0x7fdd14151780 2024-01-20T08:31:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.721+0000 7fdd18905640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd141577f0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.722+0000 7fdd12d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 0x7fdd1406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.722+0000 7fdd12d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 0x7fdd1406aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49020/0 (socket says 172.21.15.87:49020) 2024-01-20T08:31:56.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.722+0000 7fdd12d9d640 1 -- 172.21.15.87:0/1545668113 learned_addr learned my addr 172.21.15.87:0/1545668113 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:56.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.722+0000 7fdd11d9b640 1 --2- 172.21.15.87:0/1545668113 >> v2:172.21.15.116:3300/0 conn(0x7fdd14151780 0x7fdd1406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.724+0000 7fdd12d9d640 1 -- 172.21.15.87:0/1545668113 >> v2:172.21.15.116:3300/0 conn(0x7fdd14151780 msgr2=0x7fdd1406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.724+0000 7fdd12d9d640 1 --2- 172.21.15.87:0/1545668113 >> v2:172.21.15.116:3300/0 conn(0x7fdd14151780 0x7fdd1406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.724+0000 7fdd12d9d640 1 -- 172.21.15.87:0/1545668113 >> v1:172.21.15.116:6789/0 conn(0x7fdd141513a0 legacy=0x7fdd140663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.724+0000 7fdd12d9d640 1 -- 172.21.15.87:0/1545668113 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14157ad0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.727+0000 7fdd12d9d640 1 --2- 172.21.15.87:0/1545668113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 0x7fdd1406aaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009870 tx=0x7fdd0c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9d3ff0472d4f85 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:56.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.731+0000 7fdd1159a640 1 -- 172.21.15.87:0/1545668113 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c031640 con 0x7fdd1406bbc0 2024-01-20T08:31:56.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.731+0000 7fdd1159a640 1 -- 172.21.15.87:0/1545668113 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c0317a0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.731+0000 7fdd1159a640 1 -- 172.21.15.87:0/1545668113 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c031b30 con 0x7fdd1406bbc0 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 -- 172.21.15.87:0/1545668113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 msgr2=0x7fdd1406aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 --2- 172.21.15.87:0/1545668113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 0x7fdd1406aaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009870 tx=0x7fdd0c02b8f0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 -- 172.21.15.87:0/1545668113 shutdown_connections 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 --2- 172.21.15.87:0/1545668113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd1406bbc0 0x7fdd1406aaf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 --2- 172.21.15.87:0/1545668113 >> v2:172.21.15.116:3300/0 conn(0x7fdd14151780 0x7fdd1406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 -- 172.21.15.87:0/1545668113 >> 172.21.15.87:0/1545668113 conn(0x7fdd140cb310 msgr2=0x7fdd140cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 -- 172.21.15.87:0/1545668113 shutdown_connections 2024-01-20T08:31:56.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.732+0000 7fdd18905640 1 -- 172.21.15.87:0/1545668113 wait complete. 2024-01-20T08:31:56.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.733+0000 7fdd18905640 1 Processor -- start 2024-01-20T08:31:56.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.733+0000 7fdd18905640 1 -- start start 2024-01-20T08:31:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd18905640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 0x7fdd141ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd18905640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd141513a0 0x7fdd140c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd18905640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd14151780 0x7fdd140c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd18905640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd141ec3c0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd18905640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1414c360 con 0x7fdd141513a0 2024-01-20T08:31:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd18905640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1414ab90 con 0x7fdd14151780 2024-01-20T08:31:56.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.734+0000 7fdd1259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 0x7fdd141ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.735+0000 7fdd12d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd14151780 0x7fdd140c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.735+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd141513a0 0x7fdd140c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.735+0000 7fdd1259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 0x7fdd141ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34176/0 (socket says 172.21.15.87:34176) 2024-01-20T08:31:56.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.735+0000 7fdd1259c640 1 -- 172.21.15.87:0/2712568342 learned_addr learned my addr 172.21.15.87:0/2712568342 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:56.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.738+0000 7fdd1259c640 1 -- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd14151780 msgr2=0x7fdd140c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.738+0000 7fdd1259c640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd14151780 0x7fdd140c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.738+0000 7fdd1259c640 1 -- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd141513a0 msgr2=0x7fdd140c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:56.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.738+0000 7fdd1259c640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd141513a0 0x7fdd140c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:56.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.738+0000 7fdd1259c640 1 -- 172.21.15.87:0/2712568342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0094d0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.738+0000 7fdd12d9d640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd14151780 0x7fdd140c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:31:56.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.741+0000 7fdd1259c640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 0x7fdd141ebe80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c760 tx=0x7fdd0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.746+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04006d50 con 0x7fdd1406bbc0 2024-01-20T08:31:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.746+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0400ce70 con 0x7fdd1406bbc0 2024-01-20T08:31:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.746+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003b90 con 0x7fdd1406bbc0 2024-01-20T08:31:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.746+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1414a2d0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.746+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1414c5b0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.747+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd14074290 con 0x7fdd1406bbc0 2024-01-20T08:31:56.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.764+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdd04003cf0 con 0x7fdd1406bbc0 2024-01-20T08:31:56.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.765+0000 7fdd02ffd640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcf406af20 0x7fdcf406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:56.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.765+0000 7fdd11d9b640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcf406af20 0x7fdcf406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:56.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.765+0000 7fdd11d9b640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcf406af20 0x7fdcf406d3e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd141e9030 tx=0x7fdd08009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:56.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.768+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fdd04058af0 con 0x7fdd1406bbc0 2024-01-20T08:31:57.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:56.996+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdd040588c0 con 0x7fdd1406bbc0 2024-01-20T08:31:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.165+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdd1414f090 con 0x7fdd1406bbc0 2024-01-20T08:31:57.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.172+0000 7fdd02ffd640 1 -- 172.21.15.87:0/2712568342 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdd04058450 con 0x7fdd1406bbc0 2024-01-20T08:31:57.175 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcf406af20 msgr2=0x7fdcf406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcf406af20 0x7fdcf406d3e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd141e9030 tx=0x7fdd08009190 comp rx=0 tx=0).stop 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 msgr2=0x7fdd141ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 0x7fdd141ebe80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c760 tx=0x7fdd0400cc20 comp rx=0 tx=0).stop 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 shutdown_connections 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcf406af20 0x7fdcf406d3e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd14151780 0x7fdd140c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd141513a0 0x7fdd140c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 --2- 172.21.15.87:0/2712568342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd1406bbc0 0x7fdd141ebe80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 >> 172.21.15.87:0/2712568342 conn(0x7fdd140cb310 msgr2=0x7fdd140663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 shutdown_connections 2024-01-20T08:31:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.174+0000 7fdd18905640 1 -- 172.21.15.87:0/2712568342 wait complete. 2024-01-20T08:31:57.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:57.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.284+0000 7fd7d370f640 1 Processor -- start 2024-01-20T08:31:57.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.284+0000 7fd7d370f640 1 -- start start 2024-01-20T08:31:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d370f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7cc0ca260 0x7fd7cc0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d370f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc15ba40 con 0x7fd7cc070cf0 2024-01-20T08:31:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d370f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc1576e0 con 0x7fd7cc0710d0 2024-01-20T08:31:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d370f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c3bb0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d1cac640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7cc0ca260 0x7fd7cc0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d0caa640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd7cc0710d0 0x7fd7cc151350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35586/0 (socket says 172.21.15.87:35586) 2024-01-20T08:31:57.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.285+0000 7fd7d0caa640 1 -- 172.21.15.87:0/2645174928 learned_addr learned my addr 172.21.15.87:0/2645174928 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:57.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.288+0000 7fd7c3fff640 1 -- 172.21.15.87:0/2645174928 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98560577 0 0) 0x7fd7cc1576e0 con 0x7fd7cc0710d0 2024-01-20T08:31:57.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.288+0000 7fd7c3fff640 1 -- 172.21.15.87:0/2645174928 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b8003520 con 0x7fd7cc0710d0 2024-01-20T08:31:57.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.288+0000 7fd7d1cac640 1 -- 172.21.15.87:0/2645174928 >> v1:172.21.15.116:6789/0 conn(0x7fd7cc0710d0 legacy=0x7fd7cc151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.288+0000 7fd7d1cac640 1 -- 172.21.15.87:0/2645174928 >> v1:172.21.15.87:6789/0 conn(0x7fd7cc070cf0 legacy=0x7fd7cc157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.288+0000 7fd7d1cac640 1 -- 172.21.15.87:0/2645174928 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc15aec0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.291+0000 7fd7d1cac640 1 --2- 172.21.15.87:0/2645174928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc0c7680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009d90 tx=0x7fd7c402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751cb55a8b1b7c08 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.295+0000 7fd7c3fff640 1 -- 172.21.15.87:0/2645174928 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c402cd80 con 0x7fd7cc0ca260 2024-01-20T08:31:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.295+0000 7fd7c3fff640 1 -- 172.21.15.87:0/2645174928 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4034440 con 0x7fd7cc0ca260 2024-01-20T08:31:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.295+0000 7fd7c3fff640 1 -- 172.21.15.87:0/2645174928 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40347d0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 -- 172.21.15.87:0/2645174928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 msgr2=0x7fd7cc0c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 --2- 172.21.15.87:0/2645174928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc0c7680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009d90 tx=0x7fd7c402bfd0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 -- 172.21.15.87:0/2645174928 shutdown_connections 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 --2- 172.21.15.87:0/2645174928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc0c7680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 -- 172.21.15.87:0/2645174928 >> 172.21.15.87:0/2645174928 conn(0x7fd7cc066ac0 msgr2=0x7fd7cc06ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 -- 172.21.15.87:0/2645174928 shutdown_connections 2024-01-20T08:31:57.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.296+0000 7fd7d370f640 1 -- 172.21.15.87:0/2645174928 wait complete. 2024-01-20T08:31:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.297+0000 7fd7d370f640 1 Processor -- start 2024-01-20T08:31:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.297+0000 7fd7d370f640 1 -- start start 2024-01-20T08:31:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d370f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7cc0710d0 0x7fd7cc1f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d370f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d370f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7cc151710 0x7fd7cc1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d370f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc152b70 con 0x7fd7cc0ca260 2024-01-20T08:31:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d370f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc154070 con 0x7fd7cc0710d0 2024-01-20T08:31:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d370f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc1542c0 con 0x7fd7cc151710 2024-01-20T08:31:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.298+0000 7fd7d0caa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.299+0000 7fd7d14ab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7cc0710d0 0x7fd7cc1f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.299+0000 7fd7d1cac640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7cc151710 0x7fd7cc1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.299+0000 7fd7d0caa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34192/0 (socket says 172.21.15.87:34192) 2024-01-20T08:31:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.299+0000 7fd7d0caa640 1 -- 172.21.15.87:0/3674865356 learned_addr learned my addr 172.21.15.87:0/3674865356 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:57.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.302+0000 7fd7d0caa640 1 -- 172.21.15.87:0/3674865356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7cc151710 msgr2=0x7fd7cc1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.302+0000 7fd7d0caa640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7cc151710 0x7fd7cc1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.302+0000 7fd7d0caa640 1 -- 172.21.15.87:0/3674865356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7cc0710d0 msgr2=0x7fd7cc1f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.302+0000 7fd7d0caa640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7cc0710d0 0x7fd7cc1f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.302+0000 7fd7d0caa640 1 -- 172.21.15.87:0/3674865356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4009a70 con 0x7fd7cc0ca260 2024-01-20T08:31:57.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.304+0000 7fd7d0caa640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc152f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c4d0 tx=0x7fd7bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:57.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.308+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc005ce0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.308+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc005e40 con 0x7fd7cc0ca260 2024-01-20T08:31:57.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.308+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc1545a0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.308+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007270 con 0x7fd7cc0ca260 2024-01-20T08:31:57.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.308+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc152790 con 0x7fd7cc0ca260 2024-01-20T08:31:57.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.309+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc074290 con 0x7fd7cc0ca260 2024-01-20T08:31:57.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.552+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd7bc0073d0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.552+0000 7fd7c1ffb640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af00 0x7fd7b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.552+0000 7fd7d14ab640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af00 0x7fd7b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.553+0000 7fd7d14ab640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af00 0x7fd7b006d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80016e0 tx=0x7fd7c8009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:57.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.568+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fd7bc057b90 con 0x7fd7cc0ca260 2024-01-20T08:31:57.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.568+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd7bc053ed0 con 0x7fd7cc0ca260 2024-01-20T08:31:57.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.778+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7cc156b90 con 0x7fd7cc0ca260 2024-01-20T08:31:57.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.790+0000 7fd7c1ffb640 1 -- 172.21.15.87:0/3674865356 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6584 (secure 0 0 0) 0x7fd7bc017090 con 0x7fd7cc0ca260 2024-01-20T08:31:57.794 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:57.794 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:45.461258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4290,"mds_2":4287,"mds_3":4284,"mds_4":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":4,"incarnation":17,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":12,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4287":{"gid":4287,"name":"c","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4290":{"gid":4290,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-20T08:31:57.794 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 18 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af00 msgr2=0x7fd7b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af00 0x7fd7b006d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80016e0 tx=0x7fd7c8009040 comp rx=0 tx=0).stop 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 msgr2=0x7fd7cc152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc152f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c4d0 tx=0x7fd7bc00c990 comp rx=0 tx=0).stop 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 shutdown_connections 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd7b006af00 0x7fd7b006d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7cc151710 0x7fd7cc1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7cc0ca260 0x7fd7cc152f90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.792+0000 7fd7d370f640 1 --2- 172.21.15.87:0/3674865356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7cc0710d0 0x7fd7cc1f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.793+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 >> 172.21.15.87:0/3674865356 conn(0x7fd7cc070cf0 msgr2=0x7fd7cc0c22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:57.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.793+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 shutdown_connections 2024-01-20T08:31:57.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.793+0000 7fd7d370f640 1 -- 172.21.15.87:0/3674865356 wait complete. 2024-01-20T08:31:57.806 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:31:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.903+0000 7fdcd471d640 1 Processor -- start 2024-01-20T08:31:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.903+0000 7fdcd471d640 1 -- start start 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd471d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdccc0c6810 0x7fdccc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd471d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc150b00 con 0x7fdccc156a30 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd471d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0c3f00 con 0x7fdccc06fff0 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd471d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc0cbab0 con 0x7fdccc0c6810 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd1cb8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdccc0c6810 0x7fdccc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd24b9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdccc06fff0 0x7fdccc0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35592/0 (socket says 172.21.15.87:35592) 2024-01-20T08:31:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.904+0000 7fdcd24b9640 1 -- 172.21.15.87:0/433710318 learned_addr learned my addr 172.21.15.87:0/433710318 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.907+0000 7fdcd14b7640 1 -- 172.21.15.87:0/433710318 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925661907 0 0) 0x7fdcc8001110 con 0x7fdccc06fff0 2024-01-20T08:31:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.907+0000 7fdcd14b7640 1 -- 172.21.15.87:0/433710318 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb8003520 con 0x7fdccc06fff0 2024-01-20T08:31:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.908+0000 7fdcd1cb8640 1 -- 172.21.15.87:0/433710318 >> v1:172.21.15.116:6789/0 conn(0x7fdccc06fff0 legacy=0x7fdccc0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.908+0000 7fdcd1cb8640 1 -- 172.21.15.87:0/433710318 >> v1:172.21.15.87:6789/0 conn(0x7fdccc156a30 legacy=0x7fdccc160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.908+0000 7fdcd1cb8640 1 -- 172.21.15.87:0/433710318 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc1565a0 con 0x7fdccc0c6810 2024-01-20T08:31:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.911+0000 7fdcd1cb8640 1 --2- 172.21.15.87:0/433710318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc0c6810 0x7fdccc0703d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0097a0 tx=0x7fdcbc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5b819ec5cba9f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.914+0000 7fdcd14b7640 1 -- 172.21.15.87:0/433710318 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031830 con 0x7fdccc0c6810 2024-01-20T08:31:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.914+0000 7fdcd14b7640 1 -- 172.21.15.87:0/433710318 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc031990 con 0x7fdccc0c6810 2024-01-20T08:31:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.914+0000 7fdcd14b7640 1 -- 172.21.15.87:0/433710318 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031d20 con 0x7fdccc0c6810 2024-01-20T08:31:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 -- 172.21.15.87:0/433710318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc0c6810 msgr2=0x7fdccc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 --2- 172.21.15.87:0/433710318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc0c6810 0x7fdccc0703d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0097a0 tx=0x7fdcbc02b8d0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 -- 172.21.15.87:0/433710318 shutdown_connections 2024-01-20T08:31:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 --2- 172.21.15.87:0/433710318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc0c6810 0x7fdccc0703d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 -- 172.21.15.87:0/433710318 >> 172.21.15.87:0/433710318 conn(0x7fdccc06b6c0 msgr2=0x7fdccc06baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 -- 172.21.15.87:0/433710318 shutdown_connections 2024-01-20T08:31:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.915+0000 7fdcd471d640 1 -- 172.21.15.87:0/433710318 wait complete. 2024-01-20T08:31:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.916+0000 7fdcd471d640 1 Processor -- start 2024-01-20T08:31:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 -- start start 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdccc06fff0 0x7fdccc1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdccc0c6810 0x7fdccc151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 0x7fdccc0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc150f40 con 0x7fdccc156a30 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc06b310 con 0x7fdccc06fff0 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.917+0000 7fdcd471d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc158660 con 0x7fdccc0c6810 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.918+0000 7fdcd2cba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 0x7fdccc0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.918+0000 7fdcd24b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdccc06fff0 0x7fdccc1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.918+0000 7fdcd1cb8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdccc0c6810 0x7fdccc151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.918+0000 7fdcd2cba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 0x7fdccc0ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34214/0 (socket says 172.21.15.87:34214) 2024-01-20T08:31:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.918+0000 7fdcd2cba640 1 -- 172.21.15.87:0/2181085543 learned_addr learned my addr 172.21.15.87:0/2181085543 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.922+0000 7fdcd2cba640 1 -- 172.21.15.87:0/2181085543 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdccc0c6810 msgr2=0x7fdccc151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.922+0000 7fdcd2cba640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdccc0c6810 0x7fdccc151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.922+0000 7fdcd2cba640 1 -- 172.21.15.87:0/2181085543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdccc06fff0 msgr2=0x7fdccc1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.922+0000 7fdcd2cba640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdccc06fff0 0x7fdccc1597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.922+0000 7fdcd2cba640 1 -- 172.21.15.87:0/2181085543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc009400 con 0x7fdccc156a30 2024-01-20T08:31:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.924+0000 7fdcd2cba640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 0x7fdccc0ca780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c6b0 tx=0x7fdcc400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.927+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4003b70 con 0x7fdccc156a30 2024-01-20T08:31:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.927+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4003cf0 con 0x7fdccc156a30 2024-01-20T08:31:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.927+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4005310 con 0x7fdccc156a30 2024-01-20T08:31:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.927+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc1575d0 con 0x7fdccc156a30 2024-01-20T08:31:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.927+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc157b90 con 0x7fdccc156a30 2024-01-20T08:31:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.929+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc074290 con 0x7fdccc156a30 2024-01-20T08:31:57.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.947+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdcc4005470 con 0x7fdccc156a30 2024-01-20T08:31:57.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.947+0000 7fdcc2ffd640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcb006af20 0x7fdcb006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:57.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.947+0000 7fdcd24b9640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcb006af20 0x7fdcb006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:57.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.947+0000 7fdcd24b9640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcb006af20 0x7fdcb006d3e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005880 tx=0x7fdcc8009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:57.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:57.950+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fdcc4058840 con 0x7fdccc156a30 2024-01-20T08:31:58.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.179+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdcc4054b80 con 0x7fdccc156a30 2024-01-20T08:31:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.357+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdccc0cb620 con 0x7fdccc156a30 2024-01-20T08:31:58.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.398+0000 7fdcc2ffd640 1 -- 172.21.15.87:0/2181085543 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7fdcc4003e60 con 0x7fdccc156a30 2024-01-20T08:31:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:58.402 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:31:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcb006af20 msgr2=0x7fdcb006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcb006af20 0x7fdcb006d3e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005880 tx=0x7fdcc8009040 comp rx=0 tx=0).stop 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 msgr2=0x7fdccc0ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 0x7fdccc0ca780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c6b0 tx=0x7fdcc400cb70 comp rx=0 tx=0).stop 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 shutdown_connections 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fdcb006af20 0x7fdcb006d3e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdccc156a30 0x7fdccc0ca780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdccc0c6810 0x7fdccc151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 --2- 172.21.15.87:0/2181085543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdccc06fff0 0x7fdccc1597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 >> 172.21.15.87:0/2181085543 conn(0x7fdccc06b6c0 msgr2=0x7fdccc0c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:58.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 shutdown_connections 2024-01-20T08:31:58.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.400+0000 7fdcd471d640 1 -- 172.21.15.87:0/2181085543 wait complete. 2024-01-20T08:31:58.415 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:31:58.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.512+0000 7f5708a7c640 1 Processor -- start 2024-01-20T08:31:58.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.512+0000 7f5708a7c640 1 -- start start 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.513+0000 7f5708a7c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5704070cf0 0x7f57040c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.513+0000 7f5708a7c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57040710d0 0x7f5704157b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.513+0000 7f5708a7c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570415ba40 con 0x7f57040ca260 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.513+0000 7f5708a7c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57040c3bb0 con 0x7f57040710d0 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.513+0000 7f5708a7c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57040c96f0 con 0x7f5704070cf0 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.514+0000 7f5701d9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57040710d0 0x7f5704157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.514+0000 7f570259c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5704070cf0 0x7f57040c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.514+0000 7f5702d9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f57040ca260 0x7f5704151350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35594/0 (socket says 172.21.15.87:35594) 2024-01-20T08:31:58.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.514+0000 7f5702d9d640 1 -- 172.21.15.87:0/3919595556 learned_addr learned my addr 172.21.15.87:0/3919595556 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:58.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.516+0000 7f5701d9b640 1 -- 172.21.15.87:0/3919595556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5704070cf0 msgr2=0x7f57040c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:58.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.516+0000 7f5701d9b640 1 --2- 172.21.15.87:0/3919595556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5704070cf0 0x7f57040c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.516+0000 7f5701d9b640 1 -- 172.21.15.87:0/3919595556 >> v1:172.21.15.116:6789/0 conn(0x7f57040ca260 legacy=0x7f5704151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.516+0000 7f5701d9b640 1 -- 172.21.15.87:0/3919595556 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57041576e0 con 0x7f57040710d0 2024-01-20T08:31:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.518+0000 7f5701d9b640 1 --2- 172.21.15.87:0/3919595556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040710d0 0x7f5704157b70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56f80097a0 tx=0x7f56f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cac978f1b73a3a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:58.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.522+0000 7f570159a640 1 -- 172.21.15.87:0/3919595556 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8030610 con 0x7f57040710d0 2024-01-20T08:31:58.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.522+0000 7f570159a640 1 -- 172.21.15.87:0/3919595556 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8030770 con 0x7f57040710d0 2024-01-20T08:31:58.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.522+0000 7f570159a640 1 -- 172.21.15.87:0/3919595556 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8030b00 con 0x7f57040710d0 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.522+0000 7f5708a7c640 1 -- 172.21.15.87:0/3919595556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040710d0 msgr2=0x7f5704157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.522+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3919595556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040710d0 0x7f5704157b70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56f80097a0 tx=0x7f56f802b960 comp rx=0 tx=0).stop 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.523+0000 7f5708a7c640 1 -- 172.21.15.87:0/3919595556 shutdown_connections 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.523+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3919595556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040710d0 0x7f5704157b70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.523+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3919595556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5704070cf0 0x7f57040c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.523+0000 7f5708a7c640 1 -- 172.21.15.87:0/3919595556 >> 172.21.15.87:0/3919595556 conn(0x7f5704066ac0 msgr2=0x7f570406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:58.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.523+0000 7f5708a7c640 1 -- 172.21.15.87:0/3919595556 shutdown_connections 2024-01-20T08:31:58.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.523+0000 7f5708a7c640 1 -- 172.21.15.87:0/3919595556 wait complete. 2024-01-20T08:31:58.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.524+0000 7f5708a7c640 1 Processor -- start 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.524+0000 7f5708a7c640 1 -- start start 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5708a7c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57040710d0 0x7f5704159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5708a7c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57040ca260 0x7f57040cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5708a7c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 0x7f57040cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5708a7c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040cade0 con 0x7f57040cd5b0 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5708a7c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041ed080 con 0x7f57040710d0 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5708a7c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57041ed360 con 0x7f57040ca260 2024-01-20T08:31:58.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5702d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 0x7f57040cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f5701d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57040ca260 0x7f57040cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.525+0000 7f570259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57040710d0 0x7f5704159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.526+0000 7f5702d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 0x7f57040cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34232/0 (socket says 172.21.15.87:34232) 2024-01-20T08:31:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.526+0000 7f5702d9d640 1 -- 172.21.15.87:0/3648111803 learned_addr learned my addr 172.21.15.87:0/3648111803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:58.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.529+0000 7f5702d9d640 1 -- 172.21.15.87:0/3648111803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57040ca260 msgr2=0x7f57040cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:58.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.529+0000 7f5702d9d640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57040ca260 0x7f57040cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.529+0000 7f5702d9d640 1 -- 172.21.15.87:0/3648111803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57040710d0 msgr2=0x7f5704159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:58.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.529+0000 7f5702d9d640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57040710d0 0x7f5704159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:58.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.529+0000 7f5702d9d640 1 -- 172.21.15.87:0/3648111803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009400 con 0x7f57040cd5b0 2024-01-20T08:31:58.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.531+0000 7f5702d9d640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 0x7f57040cb7b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c340 tx=0x7f56fc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:58.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.534+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc005ce0 con 0x7f57040cd5b0 2024-01-20T08:31:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.534+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc005e40 con 0x7f57040cd5b0 2024-01-20T08:31:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.534+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57041ed640 con 0x7f57040cd5b0 2024-01-20T08:31:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.534+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0072d0 con 0x7f57040cd5b0 2024-01-20T08:31:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.535+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57040caa70 con 0x7f57040cd5b0 2024-01-20T08:31:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.535+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57040cac00 con 0x7f57040cd5b0 2024-01-20T08:31:58.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.778+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f56fc0074f0 con 0x7f57040cd5b0 2024-01-20T08:31:58.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.779+0000 7f56f2ffd640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56e406af20 0x7f56e406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:58.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.779+0000 7f570259c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56e406af20 0x7f56e406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:58.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.779+0000 7f570259c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56e406af20 0x7f56e406d3e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56f40113e0 tx=0x7f56f400b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:58.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.793+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f56fc057ba0 con 0x7f57040cd5b0 2024-01-20T08:31:58.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:58.794+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f56fc053ee0 con 0x7f57040cd5b0 2024-01-20T08:31:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.004+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5704074290 con 0x7f57040cd5b0 2024-01-20T08:31:59.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.018+0000 7f56f2ffd640 1 -- 172.21.15.87:0/3648111803 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6584 (secure 0 0 0) 0x7f56fc017090 con 0x7f57040cd5b0 2024-01-20T08:31:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:31:59.022 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:45.461258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4290,"mds_2":4287,"mds_3":4284,"mds_4":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":4,"incarnation":17,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":12,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4287":{"gid":4287,"name":"c","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4290":{"gid":4290,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-20T08:31:59.022 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 18 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56e406af20 msgr2=0x7f56e406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56e406af20 0x7f56e406d3e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56f40113e0 tx=0x7f56f400b040 comp rx=0 tx=0).stop 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 msgr2=0x7f57040cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 0x7f57040cb7b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c340 tx=0x7f56fc00c800 comp rx=0 tx=0).stop 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 shutdown_connections 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f56e406af20 0x7f56e406d3e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57040cd5b0 0x7f57040cb7b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57040ca260 0x7f57040cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 --2- 172.21.15.87:0/3648111803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57040710d0 0x7f5704159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 >> 172.21.15.87:0/3648111803 conn(0x7f5704070cf0 msgr2=0x7f57041569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 shutdown_connections 2024-01-20T08:31:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.020+0000 7f5708a7c640 1 -- 172.21.15.87:0/3648111803 wait complete. 2024-01-20T08:31:59.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-20T08:31:09.352915+0000', 'modified': '2024-01-20T08:31:45.461258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4293, 'mds_1': 4290, 'mds_2': 4287, 'mds_3': 4284, 'mds_4': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'f', 'rank': 4, 'incarnation': 17, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.116:6837/1361375250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 1361375250}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 1361375250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4284': {'gid': 4284, 'name': 'e', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.87:6835/890269228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 890269228}, {'type': 'v1', 'addr': '172.21.15.87:6835', 'nonce': 890269228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.87:6837/279334988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 279334988}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 279334988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4290': {'gid': 4290, 'name': 'd', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.116:6833/1888537461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 1888537461}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 1888537461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4293': {'gid': 4293, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.87:6839/3149977016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 3149977016}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 3149977016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-20T08:31:59.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-01-20T08:31:59.639 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-01-20T08:31:59.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.737+0000 7f4fdc13c640 1 Processor -- start 2024-01-20T08:31:59.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.737+0000 7f4fdc13c640 1 -- start start 2024-01-20T08:31:59.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.738+0000 7f4fdc13c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4fd40c1f30 0x7f4fd415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.738+0000 7f4fdc13c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40c6e30 con 0x7f4fd4153a40 2024-01-20T08:31:59.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.738+0000 7f4fdc13c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd415a820 con 0x7f4fd40c6760 2024-01-20T08:31:59.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.738+0000 7f4fdc13c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd406e750 con 0x7f4fd40c1f30 2024-01-20T08:31:59.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.739+0000 7f4fda6d9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4fd40c1f30 0x7f4fd415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.739+0000 7f4fd96d7640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4fd40c6760 0x7f4fd40c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35602/0 (socket says 172.21.15.87:35602) 2024-01-20T08:31:59.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.739+0000 7f4fd96d7640 1 -- 172.21.15.87:0/1651600110 learned_addr learned my addr 172.21.15.87:0/1651600110 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:31:59.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.742+0000 7f4fd8ed6640 1 -- 172.21.15.87:0/1651600110 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523506138 0 0) 0x7f4fc4001110 con 0x7f4fd40c6760 2024-01-20T08:31:59.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.742+0000 7f4fd8ed6640 1 -- 172.21.15.87:0/1651600110 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc0003520 con 0x7f4fd40c6760 2024-01-20T08:31:59.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.742+0000 7f4fda6d9640 1 -- 172.21.15.87:0/1651600110 >> v1:172.21.15.116:6789/0 conn(0x7f4fd40c6760 legacy=0x7f4fd40c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.742+0000 7f4fda6d9640 1 -- 172.21.15.87:0/1651600110 >> v1:172.21.15.87:6789/0 conn(0x7f4fd4153a40 legacy=0x7f4fd40c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.742+0000 7f4fda6d9640 1 -- 172.21.15.87:0/1651600110 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40c1510 con 0x7f4fd40c1f30 2024-01-20T08:31:59.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.746+0000 7f4fda6d9640 1 --2- 172.21.15.87:0/1651600110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd40c1f30 0x7f4fd415a2e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0097d0 tx=0x7f4fcc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f135a45caca0a1cb server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.751+0000 7f4fd8ed6640 1 -- 172.21.15.87:0/1651600110 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc030610 con 0x7f4fd40c1f30 2024-01-20T08:31:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.751+0000 7f4fd8ed6640 1 -- 172.21.15.87:0/1651600110 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc030770 con 0x7f4fd40c1f30 2024-01-20T08:31:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.751+0000 7f4fd8ed6640 1 -- 172.21.15.87:0/1651600110 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc030b00 con 0x7f4fd40c1f30 2024-01-20T08:31:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 -- 172.21.15.87:0/1651600110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd40c1f30 msgr2=0x7f4fd415a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/1651600110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd40c1f30 0x7f4fd415a2e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0097d0 tx=0x7f4fcc02b960 comp rx=0 tx=0).stop 2024-01-20T08:31:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 -- 172.21.15.87:0/1651600110 shutdown_connections 2024-01-20T08:31:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/1651600110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd40c1f30 0x7f4fd415a2e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 -- 172.21.15.87:0/1651600110 >> 172.21.15.87:0/1651600110 conn(0x7f4fd40723c0 msgr2=0x7f4fd406fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 -- 172.21.15.87:0/1651600110 shutdown_connections 2024-01-20T08:31:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.752+0000 7f4fdc13c640 1 -- 172.21.15.87:0/1651600110 wait complete. 2024-01-20T08:31:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.753+0000 7f4fdc13c640 1 Processor -- start 2024-01-20T08:31:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.753+0000 7f4fdc13c640 1 -- start start 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fdc13c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40c6760 0x7f4fd4156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fdc13c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 0x7f4fd414d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fdc13c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4fd40ca570 0x7f4fd414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fdc13c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd414e240 con 0x7f4fd4153a40 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fdc13c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41ef4b0 con 0x7f4fd40c6760 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fdc13c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41ef790 con 0x7f4fd40ca570 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.754+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 0x7f4fd414d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.755+0000 7f4fd9ed8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40c6760 0x7f4fd4156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.755+0000 7f4fda6d9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4fd40ca570 0x7f4fd414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.755+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 0x7f4fd414d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34248/0 (socket says 172.21.15.87:34248) 2024-01-20T08:31:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.755+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4122787406 learned_addr learned my addr 172.21.15.87:0/4122787406 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:59.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.758+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4122787406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4fd40ca570 msgr2=0x7f4fd414b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.758+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4fd40ca570 0x7f4fd414b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.758+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4122787406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40c6760 msgr2=0x7f4fd4156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.758+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40c6760 0x7f4fd4156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.758+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4122787406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc009400 con 0x7f4fd4153a40 2024-01-20T08:31:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.760+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 0x7f4fd414d920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c700 tx=0x7f4fc400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:59.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.764+0000 7f4fca7fc640 1 -- 172.21.15.87:0/4122787406 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4005ab0 con 0x7f4fd4153a40 2024-01-20T08:31:59.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.764+0000 7f4fca7fc640 1 -- 172.21.15.87:0/4122787406 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4005c10 con 0x7f4fd4153a40 2024-01-20T08:31:59.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.764+0000 7f4fca7fc640 1 -- 172.21.15.87:0/4122787406 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4001d50 con 0x7f4fd4153a40 2024-01-20T08:31:59.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.764+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd41efa70 con 0x7f4fd4153a40 2024-01-20T08:31:59.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.764+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41f0030 con 0x7f4fd4153a40 2024-01-20T08:31:59.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.768+0000 7f4fdeddd740 1 Processor -- start 2024-01-20T08:31:59.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.768+0000 7f4fdeddd740 1 -- start start 2024-01-20T08:31:59.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.768+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3d7f0 0x55c7f9dfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.768+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9dfc290 0x55c7f9e3e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9e3dbd0 0x55c7f9e3cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9db8010 con 0x55c7f9dfc290 2024-01-20T08:31:59.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9d69e60 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9d67550 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9e3dbd0 0x55c7f9e3cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fda6d9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3d7f0 0x55c7f9dfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fda6d9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3d7f0 0x55c7f9dfd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49074/0 (socket says 172.21.15.87:49074) 2024-01-20T08:31:59.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.769+0000 7f4fda6d9640 1 -- 172.21.15.87:0/432383595 learned_addr learned my addr 172.21.15.87:0/432383595 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.772+0000 7f4fd96d7640 1 -- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3d7f0 msgr2=0x55c7f9dfd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.772+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3d7f0 0x55c7f9dfd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.772+0000 7f4fd96d7640 1 -- 172.21.15.87:0/432383595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9dfc290 msgr2=0x55c7f9e3e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.772+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9dfc290 0x55c7f9e3e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.772+0000 7f4fd96d7640 1 -- 172.21.15.87:0/432383595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7f9d5ab00 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.774+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9e3dbd0 0x55c7f9e3cad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4001270 tx=0x7f4fc4007000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548c711ece7ced49 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.778+0000 7f4fb77fe640 1 -- 172.21.15.87:0/432383595 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc402b070 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.778+0000 7f4fb77fe640 1 -- 172.21.15.87:0/432383595 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400a930 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.778+0000 7f4fb77fe640 1 -- 172.21.15.87:0/432383595 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400aca0 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 -- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9e3dbd0 msgr2=0x55c7f9e3cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9e3dbd0 0x55c7f9e3cad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4001270 tx=0x7f4fc4007000 comp rx=0 tx=0).stop 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 -- 172.21.15.87:0/432383595 shutdown_connections 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9e3dbd0 0x55c7f9e3cad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9dfc290 0x55c7f9e3e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/432383595 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3d7f0 0x55c7f9dfd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 -- 172.21.15.87:0/432383595 >> 172.21.15.87:0/432383595 conn(0x55c7f9df9400 msgr2=0x55c7f9dfcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 -- 172.21.15.87:0/432383595 shutdown_connections 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.779+0000 7f4fdeddd740 1 -- 172.21.15.87:0/432383595 wait complete. 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.780+0000 7f4fdeddd740 1 -- start start 2024-01-20T08:31:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.780+0000 7f4fdeddd740 1 Processor -- start 2024-01-20T08:31:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e1b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9d62e40 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9d1c250 con 0x55c7f9df6a80 2024-01-20T08:31:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9d1bef0 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.781+0000 7f4fdeddd740 10 client.0 _ll_register_callbacks cb 0x55c7f9d20720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:31:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.782+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.782+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41942/0 (socket says 172.21.15.87:41942) 2024-01-20T08:31:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.782+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4191353150 learned_addr learned my addr 172.21.15.87:0/4191353150 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.783+0000 7f4fd9ed8640 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e1b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.785+0000 7f4fca7fc640 1 -- 172.21.15.87:0/4122787406 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4fc400aaa0 con 0x7f4fd4153a40 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 msgr2=0x55c7f9e1b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e1b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4191353150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 msgr2=0x55c7f9e1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4191353150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41f0030 con 0x55c7f9df6a80 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fca7fc640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4fb806ae40 0x7f4fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fda6d9640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4fb806ae40 0x7f4fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.786+0000 7f4fda6d9640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4fb806ae40 0x7f4fb806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02b9a0 tx=0x7f4fcc030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.788+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1b840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4fc408f930 tx=0x7f4fc405c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb57bb988c5f1788 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.788+0000 7f4fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-01-20T08:31:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.789+0000 7f4fca7fc640 1 -- 172.21.15.87:0/4122787406 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4fc4060050 con 0x7f4fd4153a40 2024-01-20T08:31:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.792+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/4191353150 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc402b040 con 0x55c7f9df6a80 2024-01-20T08:31:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.792+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/4191353150 <== mon.1 v2:172.21.15.116:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4fc40640f0 con 0x55c7f9df6a80 2024-01-20T08:31:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.792+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7f9d5bcd0 con 0x55c7f9df6a80 2024-01-20T08:31:59.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.794+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/4191353150 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc405fe20 con 0x55c7f9df6a80 2024-01-20T08:31:59.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.794+0000 7f4fdeddd740 10 client.4753 fetch_fsmap learned FSMap version 18 2024-01-20T08:31:59.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.794+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55c7f9d614c0 con 0x55c7f9df6a80 2024-01-20T08:31:59.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/4191353150 <== mon.1 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc408d020 con 0x55c7f9df6a80 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/4191353150 <== mon.1 v2:172.21.15.116:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f4fc4025040 con 0x55c7f9df6a80 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fdeddd740 10 client.4753 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fdeddd740 10 client.4753 resolve_mds: resolved name 'f' to daemon mds.f 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fdeddd740 20 client.4753 populate_metadata read hostname 'smithi087' 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9dfa980 0x55c7f9dfad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fdeddd740 4 client.4753 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.798+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 --> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7f9d5b9f0 con 0x55c7f9dfa980 2024-01-20T08:31:59.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.799+0000 7f4fd9ed8640 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9dfa980 0x55c7f9dfad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.839+0000 7f4fd9ed8640 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9dfa980 0x55c7f9dfad60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbbd77e9d924c91e server_cookie=b74c8ae9a738aeda in_seq=0 out_seq=0 2024-01-20T08:31:59.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.839+0000 7f4fb4ff9640 10 client.4753 ms_handle_connect on v2:172.21.15.116:6836/1361375250 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/4191353150 <== mds.4 v2:172.21.15.116:6836/1361375250 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7f9d5b9f0 con 0x55c7f9dfa980 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fb4ff9640 10 client.4753 handle_command_reply: tid=0 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 client.4753 shutdown 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 msgr2=0x55c7f9e1b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1b840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4fc408f930 tx=0x7f4fc405c300 comp rx=0 tx=0).stop 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 shutdown_connections 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9dfa980 0x55c7f9dfad60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1b840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e1b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4191353150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fb4ff9640 0 client.4753 ms_handle_reset on v2:172.21.15.116:6836/1361375250 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.945+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 >> 172.21.15.87:0/4191353150 conn(0x55c7f9dfc290 msgr2=0x55c7f9e25780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.946+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 shutdown_connections 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.946+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4191353150 wait complete. 2024-01-20T08:31:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.946+0000 7f4fdeddd740 20 client.4753 trim_cache size 0 max 16384 2024-01-20T08:31:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 Processor -- start 2024-01-20T08:31:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 -- start start 2024-01-20T08:31:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e23cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e24230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9e46520 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9ce1300 con 0x55c7f9df6a80 2024-01-20T08:31:59.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.962+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9d61fe0 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.978+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e24230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.979+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e24230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34272/0 (socket says 172.21.15.87:34272) 2024-01-20T08:31:59.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.979+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4193158500 learned_addr learned my addr 172.21.15.87:0/4193158500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:59.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.981+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4193158500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 msgr2=0x55c7f9e214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.981+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.981+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4193158500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 msgr2=0x55c7f9e23cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.981+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e23cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.981+0000 7f4fd96d7640 1 -- 172.21.15.87:0/4193158500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7f9cfe150 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.982+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e24230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005880 tx=0x7f4fc4063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370bc6306fd2d748 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:31:59.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.987+0000 7f4fb6ffd640 1 -- 172.21.15.87:0/4193158500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc405e230 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.987+0000 7f4fb6ffd640 1 -- 172.21.15.87:0/4193158500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7f9d5bcd0 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.987+0000 7f4fb6ffd640 1 -- 172.21.15.87:0/4193158500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc405fe20 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4193158500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 msgr2=0x55c7f9e24230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:31:59.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e24230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005880 tx=0x7f4fc4063000 comp rx=0 tx=0).stop 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4193158500 shutdown_connections 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e24230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/4193158500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e23cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4193158500 >> 172.21.15.87:0/4193158500 conn(0x55c7f9dfc290 msgr2=0x55c7f9df29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4193158500 shutdown_connections 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.988+0000 7f4fdeddd740 1 -- 172.21.15.87:0/4193158500 wait complete. 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.989+0000 7f4fdeddd740 1 -- start start 2024-01-20T08:31:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.989+0000 7f4fdeddd740 1 Processor -- start 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e4a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9e224c0 con 0x55c7f9e3d7f0 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9e1eec0 con 0x55c7f9df6a80 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7f9e4b160 con 0x55c7f9e3dbd0 2024-01-20T08:31:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fdeddd740 10 client.0 _ll_register_callbacks cb 0x7f4fcc009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:31:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e4a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.990+0000 7f4fda6d9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.991+0000 7f4fd96d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e4a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34286/0 (socket says 172.21.15.87:34286) 2024-01-20T08:31:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.991+0000 7f4fda6d9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e4b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49100/0 (socket says 172.21.15.87:49100) 2024-01-20T08:31:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.991+0000 7f4fd96d7640 1 -- 172.21.15.87:0/157329823 learned_addr learned my addr 172.21.15.87:0/157329823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:31:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.992+0000 7f4fd9ed8640 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:31:59.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.994+0000 7f4fd96d7640 1 -- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 msgr2=0x55c7f9e4b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.994+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e4b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.994+0000 7f4fd96d7640 1 -- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 msgr2=0x55c7f9e1e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:31:59.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.994+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:31:59.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.994+0000 7f4fd96d7640 1 -- 172.21.15.87:0/157329823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7f9d614c0 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.997+0000 7f4fd96d7640 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e4a750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4060210 tx=0x7f4fc4026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632f83a6fc1c3142 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.997+0000 7f4fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-01-20T08:32:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.999+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/157329823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc402b040 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:31:59.999+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fc400a930 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.000+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/157329823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4fc408e080 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.002+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/157329823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc405f9b0 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.002+0000 7f4fdeddd740 10 client.4713 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:00.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.002+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55c7f9d23940 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.006+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/157329823 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400a930 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/157329823 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f4fc4025070 con 0x55c7f9e3d7f0 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fdeddd740 10 client.4713 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fdeddd740 10 client.4713 resolve_mds: resolved name 'f' to daemon mds.f 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fdeddd740 20 client.4713 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9ea21e0 0x55c7f9eab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fdeddd740 4 client.4713 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-20T08:32:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 --> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c7f9a626c0 con 0x55c7f9ea21e0 2024-01-20T08:32:00.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.007+0000 7f4fd9ed8640 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9ea21e0 0x55c7f9eab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.011+0000 7f4fb4ff9640 10 client.4713 ms_handle_connect on v2:172.21.15.116:6836/1361375250 2024-01-20T08:32:00.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.011+0000 7f4fd9ed8640 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9ea21e0 0x55c7f9eab340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=245d941265d2abcd server_cookie=27dc1ae4a7635fbc in_seq=0 out_seq=0 2024-01-20T08:32:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fb4ff9640 1 -- 172.21.15.87:0/157329823 <== mds.4 v2:172.21.15.116:6836/1361375250 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c7f9a626c0 con 0x55c7f9ea21e0 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fb4ff9640 10 client.4713 handle_command_reply: tid=0 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 client.4713 shutdown 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 msgr2=0x55c7f9e4a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e4a750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4060210 tx=0x7f4fc4026000 comp rx=0 tx=0).stop 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 shutdown_connections 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:6836/1361375250,v1:172.21.15.116:6837/1361375250] conn(0x55c7f9ea21e0 0x55c7f9eab340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c7f9e3dbd0 0x55c7f9e4b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fb4ff9640 0 client.4713 ms_handle_reset on v2:172.21.15.116:6836/1361375250 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c7f9e3d7f0 0x55c7f9e4a750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 --2- 172.21.15.87:0/157329823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c7f9df6a80 0x55c7f9e1e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 >> 172.21.15.87:0/157329823 conn(0x55c7f9dfc290 msgr2=0x55c7f9e3c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 shutdown_connections 2024-01-20T08:32:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 1 -- 172.21.15.87:0/157329823 wait complete. 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdeddd740 20 client.4713 trim_cache size 0 max 16384 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "2b2378c9-6207-4be4-af26-3b0cbbc1772f", 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 4, 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4276, 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 16.126709312999999, 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 18, 2024-01-20T08:32:00.033 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 27, 2024-01-20T08:32:00.034 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 27, 2024-01-20T08:32:00.034 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 59.563417110000003 2024-01-20T08:32:00.034 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:32:00.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4fb806ae40 msgr2=0x7f4fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4fb806ae40 0x7f4fb806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02b9a0 tx=0x7f4fcc030000 comp rx=0 tx=0).stop 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 msgr2=0x7f4fd414d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.028+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 0x7f4fd414d920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c700 tx=0x7f4fc400cbc0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 shutdown_connections 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4fb806ae40 0x7f4fb806d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4fd40ca570 0x7f4fd414b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4153a40 0x7f4fd414d920 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 --2- 172.21.15.87:0/4122787406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40c6760 0x7f4fd4156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 >> 172.21.15.87:0/4122787406 conn(0x7f4fd40723c0 msgr2=0x7f4fd406ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 shutdown_connections 2024-01-20T08:32:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.029+0000 7f4fdc13c640 1 -- 172.21.15.87:0/4122787406 wait complete. 2024-01-20T08:32:00.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-01-20T08:32:00.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.139+0000 7f0d823e5640 1 Processor -- start 2024-01-20T08:32:00.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.139+0000 7f0d823e5640 1 -- start start 2024-01-20T08:32:00.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d823e5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0d7c153880 0x7f0d7c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d823e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d823e5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d7c15d100 0x7f0d7c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d823e5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c1595f0 con 0x7f0d7c15d100 2024-01-20T08:32:00.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d823e5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c15a5c0 con 0x7f0d7c153880 2024-01-20T08:32:00.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d823e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0709e0 con 0x7f0d7c0b1220 2024-01-20T08:32:00.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.140+0000 7f0d80982640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d7c15d100 0x7f0d7c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.141+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.141+0000 7f0d7bfff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0d7c153880 0x7f0d7c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.141+0000 7f0d80982640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d7c15d100 0x7f0d7c06f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34292/0 (socket says 172.21.15.87:34292) 2024-01-20T08:32:00.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.141+0000 7f0d80982640 1 -- 172.21.15.87:0/1088826921 learned_addr learned my addr 172.21.15.87:0/1088826921 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.143+0000 7f0d80982640 1 -- 172.21.15.87:0/1088826921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 msgr2=0x7f0d7c15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.143+0000 7f0d80982640 1 --2- 172.21.15.87:0/1088826921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.143+0000 7f0d80982640 1 -- 172.21.15.87:0/1088826921 >> v2:172.21.15.116:3300/0 conn(0x7f0d7c153880 msgr2=0x7f0d7c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.143+0000 7f0d80982640 1 --2- 172.21.15.87:0/1088826921 >> v2:172.21.15.116:3300/0 conn(0x7f0d7c153880 0x7f0d7c06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.143+0000 7f0d80982640 1 -- 172.21.15.87:0/1088826921 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c15a130 con 0x7f0d7c15d100 2024-01-20T08:32:00.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.145+0000 7f0d80982640 1 --2- 172.21.15.87:0/1088826921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c06f840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c072d30 tx=0x7f0d7402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70daa8688397b0c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.148+0000 7f0d7affd640 1 -- 172.21.15.87:0/1088826921 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7402cb00 con 0x7f0d7c15d100 2024-01-20T08:32:00.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d7affd640 1 -- 172.21.15.87:0/1088826921 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7402cc60 con 0x7f0d7c15d100 2024-01-20T08:32:00.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d7affd640 1 -- 172.21.15.87:0/1088826921 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740343f0 con 0x7f0d7c15d100 2024-01-20T08:32:00.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 -- 172.21.15.87:0/1088826921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 msgr2=0x7f0d7c06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1088826921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c06f840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c072d30 tx=0x7f0d7402bf60 comp rx=0 tx=0).stop 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 -- 172.21.15.87:0/1088826921 shutdown_connections 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1088826921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c06f840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1088826921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1088826921 >> v2:172.21.15.116:3300/0 conn(0x7f0d7c153880 0x7f0d7c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 -- 172.21.15.87:0/1088826921 >> 172.21.15.87:0/1088826921 conn(0x7f0d7c072390 msgr2=0x7f0d7c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 -- 172.21.15.87:0/1088826921 shutdown_connections 2024-01-20T08:32:00.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.149+0000 7f0d823e5640 1 -- 172.21.15.87:0/1088826921 wait complete. 2024-01-20T08:32:00.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.151+0000 7f0d823e5640 1 Processor -- start 2024-01-20T08:32:00.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.151+0000 7f0d823e5640 1 -- start start 2024-01-20T08:32:00.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d823e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d823e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d7c153880 0x7f0d7c144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d823e5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c1451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d823e5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15a830 con 0x7f0d7c15d100 2024-01-20T08:32:00.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d823e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15ab10 con 0x7f0d7c153880 2024-01-20T08:32:00.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d823e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15adf0 con 0x7f0d7c0b1220 2024-01-20T08:32:00.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d80982640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c1451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d7c153880 0x7f0d7c144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d80982640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c1451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34296/0 (socket says 172.21.15.87:34296) 2024-01-20T08:32:00.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.152+0000 7f0d80982640 1 -- 172.21.15.87:0/1982554888 learned_addr learned my addr 172.21.15.87:0/1982554888 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.155+0000 7f0d7bfff640 1 -- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d7c153880 msgr2=0x7f0d7c144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.156+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d7c153880 0x7f0d7c144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.156+0000 7f0d7bfff640 1 -- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 msgr2=0x7f0d7c1451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.156+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c1451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.156+0000 7f0d7bfff640 1 -- 172.21.15.87:0/1982554888 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009b30 con 0x7f0d7c0b1220 2024-01-20T08:32:00.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.158+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c14b300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00e850 tx=0x7f0d6c00ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.160+0000 7f0d78ff9640 1 -- 172.21.15.87:0/1982554888 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0095b0 con 0x7f0d7c0b1220 2024-01-20T08:32:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.160+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c15b0d0 con 0x7f0d7c0b1220 2024-01-20T08:32:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.160+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c15b690 con 0x7f0d7c0b1220 2024-01-20T08:32:00.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.161+0000 7f0d78ff9640 1 -- 172.21.15.87:0/1982554888 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c005990 con 0x7f0d7c0b1220 2024-01-20T08:32:00.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.161+0000 7f0d78ff9640 1 -- 172.21.15.87:0/1982554888 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c005d00 con 0x7f0d7c0b1220 2024-01-20T08:32:00.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 Processor -- start 2024-01-20T08:32:00.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 -- start start 2024-01-20T08:32:00.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d87f0 0x5626fd698590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd697290 0x5626fd6d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 0x5626fd6d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd653010 con 0x5626fd6d8bd0 2024-01-20T08:32:00.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd604e60 con 0x5626fd6d87f0 2024-01-20T08:32:00.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.163+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fd602550 con 0x5626fd697290 2024-01-20T08:32:00.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.164+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 0x5626fd6d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.164+0000 7f0d80982640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d87f0 0x5626fd698590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.164+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 0x5626fd6d7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34306/0 (socket says 172.21.15.87:34306) 2024-01-20T08:32:00.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.164+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/1155102019 learned_addr learned my addr 172.21.15.87:0/1155102019 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.166+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/1155102019 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd697290 msgr2=0x5626fd6d91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.166+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd697290 0x5626fd6d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.166+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/1155102019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d87f0 msgr2=0x5626fd698590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.166+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d87f0 0x5626fd698590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.166+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/1155102019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626fd5f5b00 con 0x5626fd6d8bd0 2024-01-20T08:32:00.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.168+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 0x5626fd6d7ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009bc0 tx=0x7f0d70011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ded8067afb0d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.172+0000 7f0d61ffb640 1 -- 172.21.15.87:0/1155102019 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004030 con 0x5626fd6d8bd0 2024-01-20T08:32:00.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.172+0000 7f0d61ffb640 1 -- 172.21.15.87:0/1155102019 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70004190 con 0x5626fd6d8bd0 2024-01-20T08:32:00.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.172+0000 7f0d61ffb640 1 -- 172.21.15.87:0/1155102019 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004500 con 0x5626fd6d8bd0 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 -- 172.21.15.87:0/1155102019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 msgr2=0x5626fd6d7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 0x5626fd6d7ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009bc0 tx=0x7f0d70011690 comp rx=0 tx=0).stop 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 -- 172.21.15.87:0/1155102019 shutdown_connections 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d8bd0 0x5626fd6d7ad0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd697290 0x5626fd6d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 --2- 172.21.15.87:0/1155102019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d87f0 0x5626fd698590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 -- 172.21.15.87:0/1155102019 >> 172.21.15.87:0/1155102019 conn(0x5626fd694400 msgr2=0x5626fd697ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 -- 172.21.15.87:0/1155102019 shutdown_connections 2024-01-20T08:32:00.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 -- 172.21.15.87:0/1155102019 wait complete. 2024-01-20T08:32:00.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.173+0000 7f0d84fab740 1 -- start start 2024-01-20T08:32:00.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 Processor -- start 2024-01-20T08:32:00.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d87f0 0x5626fd6de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d8bd0 0x5626fd6eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 0x5626fd6eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd5fde40 con 0x5626fd6d87f0 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd5b7250 con 0x5626fd6d5cd0 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fd5b6ef0 con 0x5626fd6d8bd0 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.174+0000 7f0d84fab740 10 client.0 _ll_register_callbacks cb 0x5626fd5bb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.175+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 0x5626fd6eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.175+0000 7f0d80982640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d87f0 0x5626fd6de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.175+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 0x5626fd6eb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41984/0 (socket says 172.21.15.87:41984) 2024-01-20T08:32:00.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.175+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/2078361360 learned_addr learned my addr 172.21.15.87:0/2078361360 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.178+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d8bd0 msgr2=0x5626fd6eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.178+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d8bd0 0x5626fd6eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.178+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d87f0 msgr2=0x5626fd6de690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.178+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d87f0 0x5626fd6de690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.178+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/2078361360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d700097c0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.181+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 0x5626fd6eb630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5626fd693640 tx=0x7f0d70004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40c4a00356e1976 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.181+0000 7f0d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-01-20T08:32:00.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.182+0000 7f0d78ff9640 1 -- 172.21.15.87:0/1982554888 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0d6c00c940 con 0x7f0d7c0b1220 2024-01-20T08:32:00.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.182+0000 7f0d78ff9640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d6806ae40 0x7f0d6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.182+0000 7f0d80982640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d6806ae40 0x7f0d6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.182+0000 7f0d80982640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d6806ae40 0x7f0d6806d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0c6890 tx=0x7f0d74041040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.184+0000 7f0d5affd640 1 -- 172.21.15.87:0/2078361360 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004030 con 0x5626fd6d5cd0 2024-01-20T08:32:00.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.184+0000 7f0d5affd640 1 -- 172.21.15.87:0/2078361360 <== mon.1 v2:172.21.15.116:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0d70003cd0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.184+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626fd5f6cd0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.185+0000 7f0d78ff9640 1 -- 172.21.15.87:0/1982554888 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0d6c058290 con 0x7f0d7c0b1220 2024-01-20T08:32:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.186+0000 7f0d5affd640 1 -- 172.21.15.87:0/2078361360 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d70003e90 con 0x5626fd6d5cd0 2024-01-20T08:32:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.186+0000 7f0d84fab740 10 client.4768 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.186+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5626fd5fc4c0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d5affd640 1 -- 172.21.15.87:0/2078361360 <== mon.1 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626fd5f6cd0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d5affd640 1 -- 172.21.15.87:0/2078361360 <== mon.1 v2:172.21.15.116:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f0d70014440 con 0x5626fd6d5cd0 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d84fab740 10 client.4768 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d84fab740 10 client.4768 resolve_mds: resolved name 'e' to daemon mds.e 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d84fab740 20 client.4768 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d84fab740 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd6ec490 0x5626fd6f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d84fab740 4 client.4768 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.190+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 --> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626fd5f69f0 con 0x5626fd6ec490 2024-01-20T08:32:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.191+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd6ec490 0x5626fd6f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.244+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd6ec490 0x5626fd6f72b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69f8be7ceb62bc14 server_cookie=f64370d73240b389 in_seq=0 out_seq=0 2024-01-20T08:32:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.244+0000 7f0d5affd640 10 client.4768 ms_handle_connect on v2:172.21.15.87:6834/890269228 2024-01-20T08:32:00.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.356+0000 7f0d5affd640 1 -- 172.21.15.87:0/2078361360 <== mds.3 v2:172.21.15.87:6834/890269228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626fd5f69f0 con 0x5626fd6ec490 2024-01-20T08:32:00.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d5affd640 10 client.4768 handle_command_reply: tid=0 2024-01-20T08:32:00.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 client.4768 shutdown 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 msgr2=0x5626fd6eb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 0x5626fd6eb630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5626fd693640 tx=0x7f0d70004730 comp rx=0 tx=0).stop 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 shutdown_connections 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd6ec490 0x5626fd6f72b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d5cd0 0x5626fd6eb630 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d8bd0 0x5626fd6eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d5affd640 0 client.4768 ms_handle_reset on v2:172.21.15.87:6834/890269228 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 --2- 172.21.15.87:0/2078361360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d87f0 0x5626fd6de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 >> 172.21.15.87:0/2078361360 conn(0x5626fd697290 msgr2=0x5626fd6922c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 shutdown_connections 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 1 -- 172.21.15.87:0/2078361360 wait complete. 2024-01-20T08:32:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.357+0000 7f0d84fab740 20 client.4768 trim_cache size 0 max 16384 2024-01-20T08:32:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.373+0000 7f0d84fab740 1 Processor -- start 2024-01-20T08:32:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.373+0000 7f0d84fab740 1 -- start start 2024-01-20T08:32:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd6d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd6eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d84fab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd57c300 con 0x5626fd6d5cd0 2024-01-20T08:32:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd5bf8d0 con 0x5626fd6d8bd0 2024-01-20T08:32:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fd6e9b00 con 0x5626fd6d87f0 2024-01-20T08:32:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d80982640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd6eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.374+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd6d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.375+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34320/0 (socket says 172.21.15.87:34320) 2024-01-20T08:32:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.375+0000 7f0d7bfff640 1 -- 172.21.15.87:0/3327089548 learned_addr learned my addr 172.21.15.87:0/3327089548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.377+0000 7f0d7bfff640 1 -- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 msgr2=0x5626fd6d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.377+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd6d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.377+0000 7f0d7bfff640 1 -- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 msgr2=0x5626fd6eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.377+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd6eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.377+0000 7f0d7bfff640 1 -- 172.21.15.87:0/3327089548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626fd599150 con 0x5626fd6d5cd0 2024-01-20T08:32:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.377+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd6d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:32:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.379+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6d74d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c054790 tx=0x7f0d6c0545d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbe7d0b49107abe server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.382+0000 7f0d617fa640 1 -- 172.21.15.87:0/3327089548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c056630 con 0x5626fd6d5cd0 2024-01-20T08:32:00.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.382+0000 7f0d617fa640 1 -- 172.21.15.87:0/3327089548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c0549a0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.382+0000 7f0d617fa640 1 -- 172.21.15.87:0/3327089548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0579c0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 -- 172.21.15.87:0/3327089548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 msgr2=0x5626fd6d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6d74d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c054790 tx=0x7f0d6c0545d0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 -- 172.21.15.87:0/3327089548 shutdown_connections 2024-01-20T08:32:00.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd6eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd6d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3327089548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6d74d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 -- 172.21.15.87:0/3327089548 >> 172.21.15.87:0/3327089548 conn(0x5626fd697290 msgr2=0x5626fd6936b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 -- 172.21.15.87:0/3327089548 shutdown_connections 2024-01-20T08:32:00.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.383+0000 7f0d84fab740 1 -- 172.21.15.87:0/3327089548 wait complete. 2024-01-20T08:32:00.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.384+0000 7f0d84fab740 1 -- start start 2024-01-20T08:32:00.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.384+0000 7f0d84fab740 1 Processor -- start 2024-01-20T08:32:00.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd73d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd73d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd6eb2c0 con 0x5626fd6d5cd0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fd743b70 con 0x5626fd6d8bd0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fd6f2b10 con 0x5626fd6d87f0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d84fab740 10 client.0 _ll_register_callbacks cb 0x7f0d70009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d80982640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd73d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd73d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d80982640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd73d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42002/0 (socket says 172.21.15.87:42002) 2024-01-20T08:32:00.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.385+0000 7f0d80982640 1 -- 172.21.15.87:0/3958724809 learned_addr learned my addr 172.21.15.87:0/3958724809 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.388+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 msgr2=0x5626fd73d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.388+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd73d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.388+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 msgr2=0x5626fd6ead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.388+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6ead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.388+0000 7f0d7b7fe640 1 -- 172.21.15.87:0/3958724809 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626fd5fc4c0 con 0x5626fd6d87f0 2024-01-20T08:32:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.390+0000 7f0d7b7fe640 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd73d140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d700116d0 tx=0x7f0d70009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0a08a6e957c7e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.390+0000 7f0d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3301/0 2024-01-20T08:32:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.392+0000 7f0d5affd640 1 -- 172.21.15.87:0/3958724809 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7003b070 con 0x5626fd6d87f0 2024-01-20T08:32:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.392+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d70004190 con 0x5626fd6d87f0 2024-01-20T08:32:00.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.395+0000 7f0d5affd640 1 -- 172.21.15.87:0/3958724809 <== mon.2 v2:172.21.15.116:3301/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f0d70000e40 con 0x5626fd6d87f0 2024-01-20T08:32:00.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.395+0000 7f0d5affd640 1 -- 172.21.15.87:0/3958724809 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d70006c10 con 0x5626fd6d87f0 2024-01-20T08:32:00.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.396+0000 7f0d84fab740 10 client.4598 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.396+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x7f0d7c15b690 con 0x5626fd6d87f0 2024-01-20T08:32:00.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d5affd640 1 -- 172.21.15.87:0/3958724809 <== mon.2 v2:172.21.15.116:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70004190 con 0x5626fd6d87f0 2024-01-20T08:32:00.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d5affd640 1 -- 172.21.15.87:0/3958724809 <== mon.2 v2:172.21.15.116:3301/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f0d7003e030 con 0x5626fd6d87f0 2024-01-20T08:32:00.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d84fab740 10 client.4598 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:00.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d84fab740 10 client.4598 resolve_mds: resolved name 'e' to daemon mds.e 2024-01-20T08:32:00.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d84fab740 20 client.4598 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:00.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd7434a0 0x5626fd746a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d84fab740 4 client.4598 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-20T08:32:00.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.407+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 --> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626fd2fd6c0 con 0x5626fd7434a0 2024-01-20T08:32:00.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.409+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd7434a0 0x5626fd746a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.412+0000 7f0d7bfff640 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd7434a0 0x5626fd746a10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=180a3f7a31726339 server_cookie=970e1ceee2e51093 in_seq=0 out_seq=0 2024-01-20T08:32:00.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.412+0000 7f0d5affd640 10 client.4598 ms_handle_connect on v2:172.21.15.87:6834/890269228 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d5affd640 1 -- 172.21.15.87:0/3958724809 <== mds.3 v2:172.21.15.87:6834/890269228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5626fd2fd6c0 con 0x5626fd7434a0 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d5affd640 10 client.4598 handle_command_reply: tid=0 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 client.4598 shutdown 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 msgr2=0x5626fd73d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd73d140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d700116d0 tx=0x7f0d70009190 comp rx=0 tx=0).stop 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 shutdown_connections 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:6834/890269228,v1:172.21.15.87:6835/890269228] conn(0x5626fd7434a0 0x5626fd746a10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626fd6d8bd0 0x5626fd73d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5626fd6d87f0 0x5626fd73d140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d5affd640 0 client.4598 ms_handle_reset on v2:172.21.15.87:6834/890269228 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 --2- 172.21.15.87:0/3958724809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626fd6d5cd0 0x5626fd6ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.429+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 >> 172.21.15.87:0/3958724809 conn(0x5626fd697290 msgr2=0x5626fd6d74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 shutdown_connections 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d84fab740 1 -- 172.21.15.87:0/3958724809 wait complete. 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d84fab740 20 client.4598 trim_cache size 0 max 16384 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "2b2378c9-6207-4be4-af26-3b0cbbc1772f", 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 3, 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4284, 2024-01-20T08:32:00.434 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 18.764405705000001, 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 18, 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 27, 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 27, 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 60.199625169000001 2024-01-20T08:32:00.435 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d6806ae40 msgr2=0x7f0d6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d6806ae40 0x7f0d6806d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0c6890 tx=0x7f0d74041040 comp rx=0 tx=0).stop 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 msgr2=0x7f0d7c14b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c14b300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00e850 tx=0x7f0d6c00ed10 comp rx=0 tx=0).stop 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 shutdown_connections 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0d6806ae40 0x7f0d6806d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d7c15d100 0x7f0d7c1451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d7c153880 0x7f0d7c144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 --2- 172.21.15.87:0/1982554888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0d7c0b1220 0x7f0d7c14b300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 >> 172.21.15.87:0/1982554888 conn(0x7f0d7c072390 msgr2=0x7f0d7c158640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 shutdown_connections 2024-01-20T08:32:00.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.430+0000 7f0d823e5640 1 -- 172.21.15.87:0/1982554888 wait complete. 2024-01-20T08:32:00.444 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-20T08:32:00.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.541+0000 7f3d624e7640 1 Processor -- start 2024-01-20T08:32:00.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.541+0000 7f3d624e7640 1 -- start start 2024-01-20T08:32:00.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.542+0000 7f3d624e7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d5c153c60 0x7f3d5c072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.542+0000 7f3d624e7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d5c15d710 0x7f3d5c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.542+0000 7f3d624e7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0c9d60 con 0x7f3d5c0cae00 2024-01-20T08:32:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.542+0000 7f3d624e7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0ca9e0 con 0x7f3d5c15d710 2024-01-20T08:32:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.542+0000 7f3d624e7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c3230 con 0x7f3d5c153c60 2024-01-20T08:32:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.543+0000 7f3d5bfff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d5c153c60 0x7f3d5c072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.543+0000 7f3d5bfff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d5c153c60 0x7f3d5c072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42006/0 (socket says 172.21.15.87:42006) 2024-01-20T08:32:00.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.543+0000 7f3d5bfff640 1 -- 172.21.15.87:0/284913625 learned_addr learned my addr 172.21.15.87:0/284913625 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.544+0000 7f3d60a84640 1 --2- 172.21.15.87:0/284913625 >> v2:172.21.15.87:3300/0 conn(0x7f3d5c15d710 0x7f3d5c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.545+0000 7f3d5bfff640 1 -- 172.21.15.87:0/284913625 >> v2:172.21.15.87:3300/0 conn(0x7f3d5c15d710 msgr2=0x7f3d5c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.545+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/284913625 >> v2:172.21.15.87:3300/0 conn(0x7f3d5c15d710 0x7f3d5c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.545+0000 7f3d5bfff640 1 -- 172.21.15.87:0/284913625 >> v1:172.21.15.87:6789/0 conn(0x7f3d5c0cae00 legacy=0x7f3d5c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.545+0000 7f3d5bfff640 1 -- 172.21.15.87:0/284913625 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0ca040 con 0x7f3d5c153c60 2024-01-20T08:32:00.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.547+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/284913625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c072680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c066660 tx=0x7f3d4c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=970df0794c18161c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.554+0000 7f3d5affd640 1 -- 172.21.15.87:0/284913625 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c030860 con 0x7f3d5c153c60 2024-01-20T08:32:00.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.554+0000 7f3d5affd640 1 -- 172.21.15.87:0/284913625 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0309c0 con 0x7f3d5c153c60 2024-01-20T08:32:00.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.554+0000 7f3d5affd640 1 -- 172.21.15.87:0/284913625 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c030d50 con 0x7f3d5c153c60 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.554+0000 7f3d624e7640 1 -- 172.21.15.87:0/284913625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 msgr2=0x7f3d5c072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.554+0000 7f3d624e7640 1 --2- 172.21.15.87:0/284913625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c072680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c066660 tx=0x7f3d4c02b9a0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.555+0000 7f3d624e7640 1 -- 172.21.15.87:0/284913625 shutdown_connections 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.555+0000 7f3d624e7640 1 --2- 172.21.15.87:0/284913625 >> v2:172.21.15.87:3300/0 conn(0x7f3d5c15d710 0x7f3d5c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.555+0000 7f3d624e7640 1 --2- 172.21.15.87:0/284913625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c072680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.555+0000 7f3d624e7640 1 -- 172.21.15.87:0/284913625 >> 172.21.15.87:0/284913625 conn(0x7f3d5c0cb310 msgr2=0x7f3d5c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.555+0000 7f3d624e7640 1 -- 172.21.15.87:0/284913625 shutdown_connections 2024-01-20T08:32:00.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.555+0000 7f3d624e7640 1 -- 172.21.15.87:0/284913625 wait complete. 2024-01-20T08:32:00.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.556+0000 7f3d624e7640 1 Processor -- start 2024-01-20T08:32:00.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.556+0000 7f3d624e7640 1 -- start start 2024-01-20T08:32:00.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d624e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d5c0cae00 0x7f3d5c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d624e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c1efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d624e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d5c15d710 0x7f3d5c15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d624e7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c156730 con 0x7f3d5c15d710 2024-01-20T08:32:00.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d624e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c15c330 con 0x7f3d5c153c60 2024-01-20T08:32:00.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d624e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1f0130 con 0x7f3d5c0cae00 2024-01-20T08:32:00.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c1efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d60a84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d5c15d710 0x7f3d5c15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d5c0cae00 0x7f3d5c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c1efbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42008/0 (socket says 172.21.15.87:42008) 2024-01-20T08:32:00.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.557+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/3420964407 learned_addr learned my addr 172.21.15.87:0/3420964407 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.561+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d5c0cae00 msgr2=0x7f3d5c15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.561+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d5c0cae00 0x7f3d5c15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.561+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d5c15d710 msgr2=0x7f3d5c15b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.561+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d5c15d710 0x7f3d5c15b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.561+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/3420964407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0094d0 con 0x7f3d5c153c60 2024-01-20T08:32:00.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.561+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d5c0cae00 0x7f3d5c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:32:00.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.563+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c1efbf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000c430 tx=0x7f3d5000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.567+0000 7f3d58ff9640 1 -- 172.21.15.87:0/3420964407 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50011790 con 0x7f3d5c153c60 2024-01-20T08:32:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.567+0000 7f3d58ff9640 1 -- 172.21.15.87:0/3420964407 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d500118f0 con 0x7f3d5c153c60 2024-01-20T08:32:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.567+0000 7f3d58ff9640 1 -- 172.21.15.87:0/3420964407 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50011ce0 con 0x7f3d5c153c60 2024-01-20T08:32:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.567+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c15ba70 con 0x7f3d5c153c60 2024-01-20T08:32:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.567+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c15c660 con 0x7f3d5c153c60 2024-01-20T08:32:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 Processor -- start 2024-01-20T08:32:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 -- start start 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed4077f0 0x5566ed3c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c6290 0x5566ed4081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed406ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed382010 con 0x5566ed3c6290 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed333e60 con 0x5566ed4077f0 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.570+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566ed331550 con 0x5566ed407bd0 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.571+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c6290 0x5566ed4081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.571+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed406ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.571+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed406ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49176/0 (socket says 172.21.15.87:49176) 2024-01-20T08:32:00.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.571+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/3751501530 learned_addr learned my addr 172.21.15.87:0/3751501530 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.573+0000 7f3d5bfff640 1 -- 172.21.15.87:0/3751501530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 msgr2=0x5566ed406ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.573+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed406ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.573+0000 7f3d5bfff640 1 -- 172.21.15.87:0/3751501530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed4077f0 msgr2=0x5566ed3c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.573+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed4077f0 0x5566ed3c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.573+0000 7f3d5bfff640 1 -- 172.21.15.87:0/3751501530 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ed324b00 con 0x5566ed3c6290 2024-01-20T08:32:00.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.575+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c6290 0x5566ed4081b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c06cb20 tx=0x7f3d4c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bd02bd3f0bd96f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d41ffb640 1 -- 172.21.15.87:0/3751501530 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c048070 con 0x5566ed3c6290 2024-01-20T08:32:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d41ffb640 1 -- 172.21.15.87:0/3751501530 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c039450 con 0x5566ed3c6290 2024-01-20T08:32:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d41ffb640 1 -- 172.21.15.87:0/3751501530 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0397e0 con 0x5566ed3c6290 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 -- 172.21.15.87:0/3751501530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c6290 msgr2=0x5566ed4081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c6290 0x5566ed4081b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c06cb20 tx=0x7f3d4c030000 comp rx=0 tx=0).stop 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 -- 172.21.15.87:0/3751501530 shutdown_connections 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed406ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c6290 0x5566ed4081b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 --2- 172.21.15.87:0/3751501530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed4077f0 0x5566ed3c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 -- 172.21.15.87:0/3751501530 >> 172.21.15.87:0/3751501530 conn(0x5566ed3c3400 msgr2=0x5566ed3c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 -- 172.21.15.87:0/3751501530 shutdown_connections 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.579+0000 7f3d65189740 1 -- 172.21.15.87:0/3751501530 wait complete. 2024-01-20T08:32:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.580+0000 7f3d65189740 1 -- start start 2024-01-20T08:32:00.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.580+0000 7f3d65189740 1 Processor -- start 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 0x5566ed41ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed32ce40 con 0x5566ed4077f0 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed2e6250 con 0x5566ed407bd0 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566ed2e5ef0 con 0x5566ed3c2080 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d65189740 10 client.0 _ll_register_callbacks cb 0x5566ed2ea720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d60a84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.581+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 0x5566ed41ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.582+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 0x5566ed41ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49188/0 (socket says 172.21.15.87:49188) 2024-01-20T08:32:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.582+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/2579396209 learned_addr learned my addr 172.21.15.87:0/2579396209 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.584+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 msgr2=0x5566ed3c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.584+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.585+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 msgr2=0x5566ed3c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.585+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3c0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.585+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/2579396209 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c0ca040 con 0x5566ed3c2080 2024-01-20T08:32:00.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.586+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 0x5566ed41ade0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d5001cc80 tx=0x7f3d50003760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77a7bc45ebd60137 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.587+0000 7f3d3affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3301/0 2024-01-20T08:32:00.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.588+0000 7f3d58ff9640 1 -- 172.21.15.87:0/3420964407 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3d50026430 con 0x7f3d5c153c60 2024-01-20T08:32:00.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.588+0000 7f3d58ff9640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d4806ae40 0x7f3d4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.589+0000 7f3d60a84640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d4806ae40 0x7f3d4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.589+0000 7f3d60a84640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d4806ae40 0x7f3d4806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d54004a80 tx=0x7f3d54009b00 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.590+0000 7f3d3affd640 1 -- 172.21.15.87:0/2579396209 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500580b0 con 0x5566ed3c2080 2024-01-20T08:32:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.590+0000 7f3d3affd640 1 -- 172.21.15.87:0/2579396209 <== mon.2 v2:172.21.15.116:3301/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f3d5005c470 con 0x5566ed3c2080 2024-01-20T08:32:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.590+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566ed325cd0 con 0x5566ed3c2080 2024-01-20T08:32:00.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.592+0000 7f3d3affd640 1 -- 172.21.15.87:0/2579396209 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d5005bb40 con 0x5566ed3c2080 2024-01-20T08:32:00.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.592+0000 7f3d58ff9640 1 -- 172.21.15.87:0/3420964407 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f3d5005bd70 con 0x7f3d5c153c60 2024-01-20T08:32:00.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.592+0000 7f3d65189740 10 client.4607 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:00.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.592+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5566ed32b4c0 con 0x5566ed3c2080 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.595+0000 7f3d3affd640 1 -- 172.21.15.87:0/2579396209 <== mon.2 v2:172.21.15.116:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5566ed325cd0 con 0x5566ed3c2080 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.595+0000 7f3d3affd640 1 -- 172.21.15.87:0/2579396209 <== mon.2 v2:172.21.15.116:3301/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f3d50073370 con 0x5566ed3c2080 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.595+0000 7f3d65189740 10 client.4607 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.596+0000 7f3d65189740 10 client.4607 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.596+0000 7f3d65189740 20 client.4607 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.596+0000 7f3d65189740 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed417d20 0x5566ed426020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.596+0000 7f3d65189740 4 client.4607 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.596+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 --> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566ed3259f0 con 0x5566ed417d20 2024-01-20T08:32:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.596+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed417d20 0x5566ed426020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.642+0000 7f3d3affd640 10 client.4607 ms_handle_connect on v2:172.21.15.87:6836/279334988 2024-01-20T08:32:00.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.642+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed417d20 0x5566ed426020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ea507180e5760bc server_cookie=77447cb1059c60bf in_seq=0 out_seq=0 2024-01-20T08:32:00.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.750+0000 7f3d3affd640 1 -- 172.21.15.87:0/2579396209 <== mds.2 v2:172.21.15.87:6836/279334988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566ed3259f0 con 0x5566ed417d20 2024-01-20T08:32:00.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.750+0000 7f3d3affd640 10 client.4607 handle_command_reply: tid=0 2024-01-20T08:32:00.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.750+0000 7f3d65189740 1 client.4607 shutdown 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.750+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 msgr2=0x5566ed41ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.750+0000 7f3d65189740 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 0x5566ed41ade0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d5001cc80 tx=0x7f3d50003760 comp rx=0 tx=0).stop 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 shutdown_connections 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed417d20 0x5566ed426020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed3c2080 0x5566ed41ade0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d3affd640 0 client.4607 ms_handle_reset on v2:172.21.15.87:6836/279334988 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 --2- 172.21.15.87:0/2579396209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 >> 172.21.15.87:0/2579396209 conn(0x5566ed3c6290 msgr2=0x5566ed3c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 shutdown_connections 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 1 -- 172.21.15.87:0/2579396209 wait complete. 2024-01-20T08:32:00.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.751+0000 7f3d65189740 20 client.4607 trim_cache size 0 max 16384 2024-01-20T08:32:00.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.767+0000 7f3d65189740 1 Processor -- start 2024-01-20T08:32:00.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.767+0000 7f3d65189740 1 -- start start 2024-01-20T08:32:00.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed3c2080 0x5566ed40ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d65189740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed3eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d65189740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed2ab300 con 0x5566ed4077f0 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed410d40 con 0x5566ed3c2080 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566ed3c3050 con 0x5566ed407bd0 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d60a84640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed3eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.768+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed3c2080 0x5566ed40ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.769+0000 7f3d60a84640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed3eda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49194/0 (socket says 172.21.15.87:49194) 2024-01-20T08:32:00.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.769+0000 7f3d60a84640 1 -- 172.21.15.87:0/2432411535 learned_addr learned my addr 172.21.15.87:0/2432411535 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.771+0000 7f3d60a84640 1 -- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed3c2080 msgr2=0x5566ed40ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.771+0000 7f3d60a84640 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed3c2080 0x5566ed40ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.771+0000 7f3d60a84640 1 -- 172.21.15.87:0/2432411535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 msgr2=0x5566ed3ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.771+0000 7f3d60a84640 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.771+0000 7f3d60a84640 1 -- 172.21.15.87:0/2432411535 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ed2c8150 con 0x5566ed407bd0 2024-01-20T08:32:00.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.771+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed3c2080 0x5566ed40ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:32:00.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.773+0000 7f3d60a84640 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed3eda40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d54004740 tx=0x7f3d5400b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb69898fce02f65 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.777+0000 7f3d417fa640 1 -- 172.21.15.87:0/2432411535 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54017430 con 0x5566ed407bd0 2024-01-20T08:32:00.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.777+0000 7f3d417fa640 1 -- 172.21.15.87:0/2432411535 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d54017590 con 0x5566ed407bd0 2024-01-20T08:32:00.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.777+0000 7f3d417fa640 1 -- 172.21.15.87:0/2432411535 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54017900 con 0x5566ed407bd0 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.777+0000 7f3d65189740 1 -- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 msgr2=0x5566ed3eda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.777+0000 7f3d65189740 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed3eda40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d54004740 tx=0x7f3d5400b050 comp rx=0 tx=0).stop 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 -- 172.21.15.87:0/2432411535 shutdown_connections 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed407bd0 0x5566ed3eda40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed4077f0 0x5566ed3ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 --2- 172.21.15.87:0/2432411535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed3c2080 0x5566ed40ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 -- 172.21.15.87:0/2432411535 >> 172.21.15.87:0/2432411535 conn(0x5566ed3c6290 msgr2=0x5566ed3bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 -- 172.21.15.87:0/2432411535 shutdown_connections 2024-01-20T08:32:00.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 -- 172.21.15.87:0/2432411535 wait complete. 2024-01-20T08:32:00.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 -- start start 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.778+0000 7f3d65189740 1 Processor -- start 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c2080 0x5566ed42a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed4077f0 0x5566ed3ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3efab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed42ae60 con 0x5566ed3c2080 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ed3e5630 con 0x5566ed407bd0 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566ed3e9c50 con 0x5566ed4077f0 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d65189740 10 client.0 _ll_register_callbacks cb 0x5566ed3c2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.779+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c2080 0x5566ed42a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.780+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed4077f0 0x5566ed3ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.780+0000 7f3d60a84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.780+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed4077f0 0x5566ed3ef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49202/0 (socket says 172.21.15.87:49202) 2024-01-20T08:32:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.780+0000 7f3d5b7fe640 1 -- 172.21.15.87:0/698690438 learned_addr learned my addr 172.21.15.87:0/698690438 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.783+0000 7f3d5bfff640 1 -- 172.21.15.87:0/698690438 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed4077f0 msgr2=0x5566ed3ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.783+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed4077f0 0x5566ed3ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.783+0000 7f3d5bfff640 1 -- 172.21.15.87:0/698690438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 msgr2=0x5566ed3efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.783+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3efab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.783+0000 7f3d5bfff640 1 -- 172.21.15.87:0/698690438 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ed324b00 con 0x5566ed3c2080 2024-01-20T08:32:00.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.785+0000 7f3d5bfff640 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c2080 0x5566ed42a920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c000c00 tx=0x7f3d4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cb88e3b1e4541f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.785+0000 7f3d3affd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-01-20T08:32:00.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.788+0000 7f3d3affd640 1 -- 172.21.15.87:0/698690438 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c048040 con 0x5566ed3c2080 2024-01-20T08:32:00.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.788+0000 7f3d3affd640 1 -- 172.21.15.87:0/698690438 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f3d4c03e040 con 0x5566ed3c2080 2024-01-20T08:32:00.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.788+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d4c039450 con 0x5566ed3c2080 2024-01-20T08:32:00.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.790+0000 7f3d3affd640 1 -- 172.21.15.87:0/698690438 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d4c04f070 con 0x5566ed3c2080 2024-01-20T08:32:00.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.790+0000 7f3d65189740 10 client.4749 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:00.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.790+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x7f3d54011040 con 0x5566ed3c2080 2024-01-20T08:32:00.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d3affd640 1 -- 172.21.15.87:0/698690438 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c039450 con 0x5566ed3c2080 2024-01-20T08:32:00.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d3affd640 1 -- 172.21.15.87:0/698690438 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f3d4c04f1d0 con 0x5566ed3c2080 2024-01-20T08:32:00.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d65189740 10 client.4749 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:00.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d65189740 10 client.4749 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-20T08:32:00.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d65189740 20 client.4749 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:00.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d65189740 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed46b890 0x5566ed46bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d65189740 4 client.4749 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-20T08:32:00.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 --> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5566ed02c6c0 con 0x5566ed46b890 2024-01-20T08:32:00.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.794+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed46b890 0x5566ed46bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.797+0000 7f3d3affd640 10 client.4749 ms_handle_connect on v2:172.21.15.87:6836/279334988 2024-01-20T08:32:00.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.797+0000 7f3d5b7fe640 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed46b890 0x5566ed46bc70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0012c94da2991bd server_cookie=6b4550a3fa4ef1af in_seq=0 out_seq=0 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d3affd640 1 -- 172.21.15.87:0/698690438 <== mds.2 v2:172.21.15.87:6836/279334988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5566ed02c6c0 con 0x5566ed46b890 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d3affd640 10 client.4749 handle_command_reply: tid=0 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 client.4749 shutdown 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c2080 msgr2=0x5566ed42a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c2080 0x5566ed42a920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c000c00 tx=0x7f3d4c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 shutdown_connections 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:6836/279334988,v1:172.21.15.87:6837/279334988] conn(0x5566ed46b890 0x5566ed46bc70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566ed407bd0 0x5566ed3efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d3affd640 0 client.4749 ms_handle_reset on v2:172.21.15.87:6836/279334988 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5566ed4077f0 0x5566ed3ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 --2- 172.21.15.87:0/698690438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566ed3c2080 0x5566ed42a920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 >> 172.21.15.87:0/698690438 conn(0x5566ed3c6290 msgr2=0x5566ed410340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 shutdown_connections 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.814+0000 7f3d65189740 1 -- 172.21.15.87:0/698690438 wait complete. 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d65189740 20 client.4749 trim_cache size 0 max 16384 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "2b2378c9-6207-4be4-af26-3b0cbbc1772f", 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 2, 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4287, 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-01-20T08:32:00.819 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-01-20T08:32:00.820 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 21.64727152, 2024-01-20T08:32:00.820 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 18, 2024-01-20T08:32:00.820 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 27, 2024-01-20T08:32:00.820 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 27, 2024-01-20T08:32:00.820 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 60.551230109000002 2024-01-20T08:32:00.820 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d4806ae40 msgr2=0x7f3d4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d4806ae40 0x7f3d4806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d54004a80 tx=0x7f3d54009b00 comp rx=0 tx=0).stop 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 msgr2=0x7f3d5c1efbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c1efbf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000c430 tx=0x7f3d5000c8f0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 shutdown_connections 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d4806ae40 0x7f3d4806d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d5c15d710 0x7f3d5c15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d5c153c60 0x7f3d5c1efbf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 --2- 172.21.15.87:0/3420964407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d5c0cae00 0x7f3d5c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 >> 172.21.15.87:0/3420964407 conn(0x7f3d5c0cb310 msgr2=0x7f3d5c0c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 shutdown_connections 2024-01-20T08:32:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.815+0000 7f3d624e7640 1 -- 172.21.15.87:0/3420964407 wait complete. 2024-01-20T08:32:00.829 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-20T08:32:00.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.925+0000 7f7485125640 1 Processor -- start 2024-01-20T08:32:00.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.925+0000 7f7485125640 1 -- start start 2024-01-20T08:32:00.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f7485125640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f74801505f0 0x7f74800c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f7485125640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 0x7f74801599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f7485125640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f74800c7dc0 0x7f74800c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f7485125640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74800c8710 con 0x7f74800c7dc0 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f7485125640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74800c7630 con 0x7f74801505f0 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f7485125640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7480071f20 con 0x7f74800c71c0 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f747e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 0x7f74801599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f747ed9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f74801505f0 0x7f74800c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.926+0000 7f747f59e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f74800c7dc0 0x7f74800c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.927+0000 7f747e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 0x7f74801599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49206/0 (socket says 172.21.15.87:49206) 2024-01-20T08:32:00.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.927+0000 7f747e59c640 1 -- 172.21.15.87:0/1883791138 learned_addr learned my addr 172.21.15.87:0/1883791138 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.929+0000 7f747e59c640 1 -- 172.21.15.87:0/1883791138 >> v2:172.21.15.116:3300/0 conn(0x7f74801505f0 msgr2=0x7f74800c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.929+0000 7f747e59c640 1 --2- 172.21.15.87:0/1883791138 >> v2:172.21.15.116:3300/0 conn(0x7f74801505f0 0x7f74800c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.929+0000 7f747e59c640 1 -- 172.21.15.87:0/1883791138 >> v2:172.21.15.87:3300/0 conn(0x7f74800c7dc0 msgr2=0x7f74800c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:00.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.929+0000 7f747e59c640 1 --2- 172.21.15.87:0/1883791138 >> v2:172.21.15.87:3300/0 conn(0x7f74800c7dc0 0x7f74800c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.929+0000 7f747e59c640 1 -- 172.21.15.87:0/1883791138 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480071c40 con 0x7f74800c71c0 2024-01-20T08:32:00.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.931+0000 7f747e59c640 1 --2- 172.21.15.87:0/1883791138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 0x7f74801599b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74800cbe50 tx=0x7f747402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f44f536536b2d93 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.934+0000 7f747dd9b640 1 -- 172.21.15.87:0/1883791138 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474031870 con 0x7f74800c71c0 2024-01-20T08:32:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.934+0000 7f747dd9b640 1 -- 172.21.15.87:0/1883791138 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74740319d0 con 0x7f74800c71c0 2024-01-20T08:32:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.934+0000 7f747dd9b640 1 -- 172.21.15.87:0/1883791138 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474031d60 con 0x7f74800c71c0 2024-01-20T08:32:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 -- 172.21.15.87:0/1883791138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 msgr2=0x7f74801599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 --2- 172.21.15.87:0/1883791138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 0x7f74801599b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74800cbe50 tx=0x7f747402b910 comp rx=0 tx=0).stop 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 -- 172.21.15.87:0/1883791138 shutdown_connections 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 --2- 172.21.15.87:0/1883791138 >> v2:172.21.15.87:3300/0 conn(0x7f74800c7dc0 0x7f74800c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 --2- 172.21.15.87:0/1883791138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74800c71c0 0x7f74801599b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 --2- 172.21.15.87:0/1883791138 >> v2:172.21.15.116:3300/0 conn(0x7f74801505f0 0x7f74800c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 -- 172.21.15.87:0/1883791138 >> 172.21.15.87:0/1883791138 conn(0x7f748006a9a0 msgr2=0x7f748006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 -- 172.21.15.87:0/1883791138 shutdown_connections 2024-01-20T08:32:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.935+0000 7f7485125640 1 -- 172.21.15.87:0/1883791138 wait complete. 2024-01-20T08:32:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 Processor -- start 2024-01-20T08:32:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 -- start start 2024-01-20T08:32:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74800c71c0 0x7f74801e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 0x7f74801f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74801505f0 0x7f74801f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c9150 con 0x7f74800c71c0 2024-01-20T08:32:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801f38d0 con 0x7f74800c7dc0 2024-01-20T08:32:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.937+0000 7f7485125640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74801f2a90 con 0x7f74801505f0 2024-01-20T08:32:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.938+0000 7f747e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 0x7f74801f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.939+0000 7f747e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 0x7f74801f0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42068/0 (socket says 172.21.15.87:42068) 2024-01-20T08:32:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.939+0000 7f747e59c640 1 -- 172.21.15.87:0/1046685778 learned_addr learned my addr 172.21.15.87:0/1046685778 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.939+0000 7f747ed9d640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74800c71c0 0x7f74801e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.939+0000 7f747f59e640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74801505f0 0x7f74801f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.943+0000 7f747e59c640 1 -- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74801505f0 msgr2=0x7f74801f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.943+0000 7f747e59c640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74801505f0 0x7f74801f11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.943+0000 7f747e59c640 1 -- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74800c71c0 msgr2=0x7f74801e0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:00.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.943+0000 7f747e59c640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74800c71c0 0x7f74801e0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:00.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.943+0000 7f747e59c640 1 -- 172.21.15.87:0/1046685778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740094d0 con 0x7f74800c7dc0 2024-01-20T08:32:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.945+0000 7f747e59c640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 0x7f74801f0c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f747402b950 tx=0x7f74740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:00.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.949+0000 7f74677fe640 1 -- 172.21.15.87:0/1046685778 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747403ac60 con 0x7f74800c7dc0 2024-01-20T08:32:00.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.949+0000 7f74677fe640 1 -- 172.21.15.87:0/1046685778 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747403adc0 con 0x7f74800c7dc0 2024-01-20T08:32:00.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.949+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74801f2d50 con 0x7f74800c7dc0 2024-01-20T08:32:00.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.949+0000 7f74677fe640 1 -- 172.21.15.87:0/1046685778 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474044630 con 0x7f74800c7dc0 2024-01-20T08:32:00.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.949+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74801f3310 con 0x7f74800c7dc0 2024-01-20T08:32:00.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.952+0000 7f7487dc1740 1 Processor -- start 2024-01-20T08:32:00.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 -- start start 2024-01-20T08:32:00.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d17c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d17c4290 0x55f0d18061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d1804ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1780010 con 0x55f0d1805bd0 2024-01-20T08:32:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1731e60 con 0x55f0d17c4290 2024-01-20T08:32:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.953+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0d172f550 con 0x55f0d18057f0 2024-01-20T08:32:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.956+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d17c4290 0x55f0d18061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.956+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d17c4290 0x55f0d18061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42074/0 (socket says 172.21.15.87:42074) 2024-01-20T08:32:00.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.956+0000 7f747ed9d640 1 -- 172.21.15.87:0/1571155845 learned_addr learned my addr 172.21.15.87:0/1571155845 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:00.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.958+0000 7f747f59e640 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d17c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:00.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:00.994+0000 7f747e59c640 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d1804ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.001+0000 7f747f59e640 1 -- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d17c4290 msgr2=0x55f0d18061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.001+0000 7f747f59e640 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d17c4290 0x55f0d18061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.001+0000 7f747f59e640 1 -- 172.21.15.87:0/1571155845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 msgr2=0x55f0d1804ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.001+0000 7f747f59e640 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d1804ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.001+0000 7f747f59e640 1 -- 172.21.15.87:0/1571155845 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1722b00 con 0x55f0d18057f0 2024-01-20T08:32:01.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.005+0000 7f747f59e640 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d17c5590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7478009aa0 tx=0x7f7478009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0fe06eb2717390 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.012+0000 7f7464ff9640 1 -- 172.21.15.87:0/1571155845 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478011d20 con 0x55f0d18057f0 2024-01-20T08:32:01.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.012+0000 7f7464ff9640 1 -- 172.21.15.87:0/1571155845 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478011e80 con 0x55f0d18057f0 2024-01-20T08:32:01.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.012+0000 7f7464ff9640 1 -- 172.21.15.87:0/1571155845 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478004230 con 0x55f0d18057f0 2024-01-20T08:32:01.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 -- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 msgr2=0x55f0d17c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d17c5590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7478009aa0 tx=0x7f7478009f60 comp rx=0 tx=0).stop 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 -- 172.21.15.87:0/1571155845 shutdown_connections 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d1804ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d17c4290 0x55f0d18061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 --2- 172.21.15.87:0/1571155845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d17c5590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 -- 172.21.15.87:0/1571155845 >> 172.21.15.87:0/1571155845 conn(0x55f0d17c1400 msgr2=0x55f0d17c4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 -- 172.21.15.87:0/1571155845 shutdown_connections 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.013+0000 7f7487dc1740 1 -- 172.21.15.87:0/1571155845 wait complete. 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.014+0000 7f7487dc1740 1 -- start start 2024-01-20T08:32:01.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.014+0000 7f7487dc1740 1 Processor -- start 2024-01-20T08:32:01.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.014+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d18057f0 0x55f0d17edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.014+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d1805bd0 0x55f0d1818c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d180c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f7487dc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d172ae40 con 0x55f0d18057f0 2024-01-20T08:32:01.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d16e4250 con 0x55f0d1805bd0 2024-01-20T08:32:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0d16e3ef0 con 0x55f0d180d340 2024-01-20T08:32:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f7487dc1740 10 client.0 _ll_register_callbacks cb 0x55f0d16e8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f747f59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d18057f0 0x55f0d17edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f747e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d180c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f747e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d180c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49240/0 (socket says 172.21.15.87:49240) 2024-01-20T08:32:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.015+0000 7f747e59c640 1 -- 172.21.15.87:0/4227587950 learned_addr learned my addr 172.21.15.87:0/4227587950 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.018+0000 7f747e59c640 1 -- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d1805bd0 msgr2=0x55f0d1818c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.018+0000 7f747e59c640 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d1805bd0 0x55f0d1818c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.018+0000 7f747e59c640 1 -- 172.21.15.87:0/4227587950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d18057f0 msgr2=0x55f0d17edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.018+0000 7f747e59c640 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d18057f0 0x55f0d17edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.018+0000 7f747e59c640 1 -- 172.21.15.87:0/4227587950 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7478009720 con 0x55f0d180d340 2024-01-20T08:32:01.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.020+0000 7f747e59c640 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d180c980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f747404d700 tx=0x7f7474031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1538afe8a4672a20 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.020+0000 7f745dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3301/0 2024-01-20T08:32:01.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.021+0000 7f74677fe640 1 -- 172.21.15.87:0/1046685778 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7474056630 con 0x7f74800c7dc0 2024-01-20T08:32:01.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.022+0000 7f74677fe640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f746c06aec0 0x7f746c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.022+0000 7f747f59e640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f746c06aec0 0x7f746c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.022+0000 7f747f59e640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f746c06aec0 0x7f746c06d380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7478009a40 tx=0x7f7478021040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.024+0000 7f745dffb640 1 -- 172.21.15.87:0/4227587950 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474079c60 con 0x55f0d180d340 2024-01-20T08:32:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.024+0000 7f745dffb640 1 -- 172.21.15.87:0/4227587950 <== mon.2 v2:172.21.15.116:3301/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7474052040 con 0x55f0d180d340 2024-01-20T08:32:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.024+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0d1723cd0 con 0x55f0d180d340 2024-01-20T08:32:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.024+0000 7f74677fe640 1 -- 172.21.15.87:0/1046685778 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f747402d040 con 0x7f74800c7dc0 2024-01-20T08:32:01.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.130+0000 7f745dffb640 1 -- 172.21.15.87:0/4227587950 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747407d280 con 0x55f0d180d340 2024-01-20T08:32:01.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.130+0000 7f7487dc1740 10 client.4625 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:01.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.130+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55f0d17294c0 con 0x55f0d180d340 2024-01-20T08:32:01.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f745dffb640 1 -- 172.21.15.87:0/4227587950 <== mon.2 v2:172.21.15.116:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f0d1723cd0 con 0x55f0d180d340 2024-01-20T08:32:01.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f745dffb640 1 -- 172.21.15.87:0/4227587950 <== mon.2 v2:172.21.15.116:3301/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f747407a030 con 0x55f0d180d340 2024-01-20T08:32:01.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f7487dc1740 10 client.4625 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:01.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f7487dc1740 10 client.4625 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-20T08:32:01.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f7487dc1740 20 client.4625 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:01.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d17ec960 0x55f0d1800480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f7487dc1740 4 client.4625 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-20T08:32:01.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.134+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 --> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0d17239f0 con 0x55f0d17ec960 2024-01-20T08:32:01.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.135+0000 7f747ed9d640 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d17ec960 0x55f0d1800480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.182+0000 7f745dffb640 10 client.4625 ms_handle_connect on v2:172.21.15.116:6832/1888537461 2024-01-20T08:32:01.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.182+0000 7f747ed9d640 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d17ec960 0x55f0d1800480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=743e546b7f39290e server_cookie=d91384e8bcab1582 in_seq=0 out_seq=0 2024-01-20T08:32:01.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f745dffb640 1 -- 172.21.15.87:0/4227587950 <== mds.1 v2:172.21.15.116:6832/1888537461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0d17239f0 con 0x55f0d17ec960 2024-01-20T08:32:01.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f745dffb640 10 client.4625 handle_command_reply: tid=0 2024-01-20T08:32:01.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 client.4625 shutdown 2024-01-20T08:32:01.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 msgr2=0x55f0d180c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d180c980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f747404d700 tx=0x7f7474031000 comp rx=0 tx=0).stop 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 shutdown_connections 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d17ec960 0x55f0d1800480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d180c980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d1805bd0 0x55f0d1818c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4227587950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d18057f0 0x55f0d17edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f745dffb640 0 client.4625 ms_handle_reset on v2:172.21.15.116:6832/1888537461 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 >> 172.21.15.87:0/4227587950 conn(0x55f0d17c4290 msgr2=0x55f0d17beb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 shutdown_connections 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 1 -- 172.21.15.87:0/4227587950 wait complete. 2024-01-20T08:32:01.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.290+0000 7f7487dc1740 20 client.4625 trim_cache size 0 max 16384 2024-01-20T08:32:01.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.305+0000 7f7487dc1740 1 Processor -- start 2024-01-20T08:32:01.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.305+0000 7f7487dc1740 1 -- start start 2024-01-20T08:32:01.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d18057f0 0x55f0d180a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f7487dc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d16a9300 con 0x55f0d1805bd0 2024-01-20T08:32:01.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d18281f0 con 0x55f0d18057f0 2024-01-20T08:32:01.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0d17c3800 con 0x55f0d180d340 2024-01-20T08:32:01.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f747e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.306+0000 7f747f59e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.307+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d18057f0 0x55f0d180a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.307+0000 7f747f59e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49256/0 (socket says 172.21.15.87:49256) 2024-01-20T08:32:01.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.307+0000 7f747f59e640 1 -- 172.21.15.87:0/4247518370 learned_addr learned my addr 172.21.15.87:0/4247518370 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.309+0000 7f747e59c640 1 -- 172.21.15.87:0/4247518370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 msgr2=0x55f0d17c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.309+0000 7f747e59c640 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.309+0000 7f747e59c640 1 -- 172.21.15.87:0/4247518370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d18057f0 msgr2=0x55f0d180a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.309+0000 7f747e59c640 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d18057f0 0x55f0d180a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.309+0000 7f747e59c640 1 -- 172.21.15.87:0/4247518370 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d16c6150 con 0x55f0d1805bd0 2024-01-20T08:32:01.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.311+0000 7f747e59c640 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55f0d17c0ce0 tx=0x7f747400b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3161699722f7c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f745ffff640 1 -- 172.21.15.87:0/4247518370 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474079c60 con 0x55f0d1805bd0 2024-01-20T08:32:01.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f745ffff640 1 -- 172.21.15.87:0/4247518370 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747407d460 con 0x55f0d1805bd0 2024-01-20T08:32:01.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f745ffff640 1 -- 172.21.15.87:0/4247518370 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474081a70 con 0x55f0d1805bd0 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 -- 172.21.15.87:0/4247518370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 msgr2=0x55f0d17c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55f0d17c0ce0 tx=0x7f747400b020 comp rx=0 tx=0).stop 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 -- 172.21.15.87:0/4247518370 shutdown_connections 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 --2- 172.21.15.87:0/4247518370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d18057f0 0x55f0d180a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 -- 172.21.15.87:0/4247518370 >> 172.21.15.87:0/4247518370 conn(0x55f0d17c4290 msgr2=0x55f0d17ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 -- 172.21.15.87:0/4247518370 shutdown_connections 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.315+0000 7f7487dc1740 1 -- 172.21.15.87:0/4247518370 wait complete. 2024-01-20T08:32:01.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.316+0000 7f7487dc1740 1 -- start start 2024-01-20T08:32:01.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.316+0000 7f7487dc1740 1 Processor -- start 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d1804b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1805070 con 0x55f0d1805bd0 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d17e5a50 con 0x55f0d180d340 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1828650 con 0x55f0d18057f0 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.317+0000 7f7487dc1740 10 client.0 _ll_register_callbacks cb 0x7f74780094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.318+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d1804b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.318+0000 7f747e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.318+0000 7f747f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.318+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d1804b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49258/0 (socket says 172.21.15.87:49258) 2024-01-20T08:32:01.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.318+0000 7f747ed9d640 1 -- 172.21.15.87:0/341648595 learned_addr learned my addr 172.21.15.87:0/341648595 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.321+0000 7f747e59c640 1 -- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 msgr2=0x55f0d1804b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.321+0000 7f747e59c640 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d1804b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.321+0000 7f747e59c640 1 -- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d180d340 msgr2=0x55f0d17c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.321+0000 7f747e59c640 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.321+0000 7f747e59c640 1 -- 172.21.15.87:0/341648595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0d17294c0 con 0x55f0d1805bd0 2024-01-20T08:32:01.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.323+0000 7f747e59c640 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f747407e1d0 tx=0x7f7474031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158f7079a2050763 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.323+0000 7f745dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-01-20T08:32:01.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.326+0000 7f745dffb640 1 -- 172.21.15.87:0/341648595 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474079c60 con 0x55f0d1805bd0 2024-01-20T08:32:01.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.326+0000 7f745dffb640 1 -- 172.21.15.87:0/341648595 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7474052040 con 0x55f0d1805bd0 2024-01-20T08:32:01.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.326+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7478011e80 con 0x55f0d1805bd0 2024-01-20T08:32:01.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.328+0000 7f745dffb640 1 -- 172.21.15.87:0/341648595 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7474085c20 con 0x55f0d1805bd0 2024-01-20T08:32:01.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.328+0000 7f7487dc1740 10 client.4767 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:01.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.328+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55f0d16eb940 con 0x55f0d1805bd0 2024-01-20T08:32:01.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.329+0000 7f745dffb640 1 -- 172.21.15.87:0/341648595 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478011e80 con 0x55f0d1805bd0 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.331+0000 7f745dffb640 1 -- 172.21.15.87:0/341648595 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f747407a030 con 0x55f0d1805bd0 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.331+0000 7f7487dc1740 10 client.4767 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.331+0000 7f7487dc1740 10 client.4767 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.332+0000 7f7487dc1740 20 client.4767 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.332+0000 7f7487dc1740 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d186afc0 0x55f0d1817640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.332+0000 7f7487dc1740 4 client.4767 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-20T08:32:01.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.332+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 --> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f0d142a6c0 con 0x55f0d186afc0 2024-01-20T08:32:01.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.332+0000 7f747ed9d640 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d186afc0 0x55f0d1817640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.335+0000 7f747ed9d640 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d186afc0 0x55f0d1817640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=432bd2ead79a42c8 server_cookie=8903b877827ad121 in_seq=0 out_seq=0 2024-01-20T08:32:01.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.335+0000 7f745dffb640 10 client.4767 ms_handle_connect on v2:172.21.15.116:6832/1888537461 2024-01-20T08:32:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f745dffb640 1 -- 172.21.15.87:0/341648595 <== mds.1 v2:172.21.15.116:6832/1888537461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f0d142a6c0 con 0x55f0d186afc0 2024-01-20T08:32:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f745dffb640 10 client.4767 handle_command_reply: tid=0 2024-01-20T08:32:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 client.4767 shutdown 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 msgr2=0x55f0d17c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f747407e1d0 tx=0x7f7474031000 comp rx=0 tx=0).stop 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 shutdown_connections 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:6832/1888537461,v1:172.21.15.116:6833/1888537461] conn(0x55f0d186afc0 0x55f0d1817640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f0d180d340 0x55f0d17c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0d1805bd0 0x55f0d17c47a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f745dffb640 0 client.4767 ms_handle_reset on v2:172.21.15.116:6832/1888537461 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 --2- 172.21.15.87:0/341648595 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f0d18057f0 0x55f0d1804b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 >> 172.21.15.87:0/341648595 conn(0x55f0d17c4290 msgr2=0x55f0d180c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 shutdown_connections 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 1 -- 172.21.15.87:0/341648595 wait complete. 2024-01-20T08:32:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7487dc1740 20 client.4767 trim_cache size 0 max 16384 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "2b2378c9-6207-4be4-af26-3b0cbbc1772f", 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 1, 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4290, 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 24.307543427999999, 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 18, 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 27, 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 27, 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 61.052869915000002 2024-01-20T08:32:01.358 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:32:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f746c06aec0 msgr2=0x7f746c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f746c06aec0 0x7f746c06d380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7478009a40 tx=0x7f7478021040 comp rx=0 tx=0).stop 2024-01-20T08:32:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 msgr2=0x7f74801f0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 0x7f74801f0c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f747402b950 tx=0x7f74740046c0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 shutdown_connections 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f746c06aec0 0x7f746c06d380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74801505f0 0x7f74801f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800c7dc0 0x7f74801f0c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 --2- 172.21.15.87:0/1046685778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74800c71c0 0x7f74801e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 >> 172.21.15.87:0/1046685778 conn(0x7f748006a9a0 msgr2=0x7f748014f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.353+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 shutdown_connections 2024-01-20T08:32:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.354+0000 7f7485125640 1 -- 172.21.15.87:0/1046685778 wait complete. 2024-01-20T08:32:01.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-20T08:32:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.465+0000 7f99a7a61640 1 Processor -- start 2024-01-20T08:32:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.465+0000 7f99a7a61640 1 -- start start 2024-01-20T08:32:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.466+0000 7f99a7a61640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f99a00b1220 0x7f99a006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.466+0000 7f99a7a61640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a00b0c60 0x7f99a006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.466+0000 7f99a7a61640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a00709e0 con 0x7f99a0153880 2024-01-20T08:32:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.466+0000 7f99a7a61640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a01595f0 con 0x7f99a00b1220 2024-01-20T08:32:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.466+0000 7f99a7a61640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a015a5c0 con 0x7f99a00b0c60 2024-01-20T08:32:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.467+0000 7f99a4ffc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f99a00b1220 0x7f99a006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.467+0000 7f99a57fd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f99a0153880 0x7f99a015cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35610/0 (socket says 172.21.15.87:35610) 2024-01-20T08:32:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.467+0000 7f99a57fd640 1 -- 172.21.15.87:0/1958185641 learned_addr learned my addr 172.21.15.87:0/1958185641 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:32:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.467+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/1958185641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a00b0c60 0x7f99a006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.469+0000 7f9997fff640 1 -- 172.21.15.87:0/1958185641 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950250235 0 0) 0x7f999c001110 con 0x7f99a0153880 2024-01-20T08:32:01.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.469+0000 7f9997fff640 1 -- 172.21.15.87:0/1958185641 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998c003520 con 0x7f99a0153880 2024-01-20T08:32:01.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.469+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1958185641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a00b0c60 msgr2=0x7f99a006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.469+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1958185641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a00b0c60 0x7f99a006f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.469+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1958185641 >> v1:172.21.15.116:6789/0 conn(0x7f99a0153880 legacy=0x7f99a015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.469+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1958185641 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a015a130 con 0x7f99a00b1220 2024-01-20T08:32:01.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.471+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1958185641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a006df10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99900097a0 tx=0x7f999002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3748fa64533a8a8f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f9997fff640 1 -- 172.21.15.87:0/1958185641 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990030610 con 0x7f99a00b1220 2024-01-20T08:32:01.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f9997fff640 1 -- 172.21.15.87:0/1958185641 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990030770 con 0x7f99a00b1220 2024-01-20T08:32:01.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f9997fff640 1 -- 172.21.15.87:0/1958185641 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990030b00 con 0x7f99a00b1220 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f99a7a61640 1 -- 172.21.15.87:0/1958185641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 msgr2=0x7f99a006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1958185641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a006df10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99900097a0 tx=0x7f999002b960 comp rx=0 tx=0).stop 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f99a7a61640 1 -- 172.21.15.87:0/1958185641 shutdown_connections 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1958185641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a00b0c60 0x7f99a006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1958185641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a006df10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.475+0000 7f99a7a61640 1 -- 172.21.15.87:0/1958185641 >> 172.21.15.87:0/1958185641 conn(0x7f99a0072390 msgr2=0x7f99a006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.476+0000 7f99a7a61640 1 -- 172.21.15.87:0/1958185641 shutdown_connections 2024-01-20T08:32:01.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.476+0000 7f99a7a61640 1 -- 172.21.15.87:0/1958185641 wait complete. 2024-01-20T08:32:01.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.477+0000 7f99a7a61640 1 Processor -- start 2024-01-20T08:32:01.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.477+0000 7f99a7a61640 1 -- start start 2024-01-20T08:32:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a7a61640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99a00b0c60 0x7f99a00c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a7a61640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a01efc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a7a61640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 0x7f99a014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a7a61640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01f01c0 con 0x7f99a00b1220 2024-01-20T08:32:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a7a61640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c4340 con 0x7f99a00b0c60 2024-01-20T08:32:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a7a61640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c5e60 con 0x7f99a0153880 2024-01-20T08:32:01.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a01efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a57fd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99a00b0c60 0x7f99a00c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.478+0000 7f99a5ffe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 0x7f99a014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.479+0000 7f99a5ffe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 0x7f99a014eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49270/0 (socket says 172.21.15.87:49270) 2024-01-20T08:32:01.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.479+0000 7f99a5ffe640 1 -- 172.21.15.87:0/1399860268 learned_addr learned my addr 172.21.15.87:0/1399860268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.482+0000 7f99a5ffe640 1 -- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99a00b0c60 msgr2=0x7f99a00c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.482+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99a00b0c60 0x7f99a00c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.482+0000 7f99a5ffe640 1 -- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 msgr2=0x7f99a01efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.482+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a01efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.482+0000 7f99a5ffe640 1 -- 172.21.15.87:0/1399860268 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990009400 con 0x7f99a0153880 2024-01-20T08:32:01.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.484+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 0x7f99a014eb20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f999800efc0 tx=0x7f999800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.488+0000 7f9995ffb640 1 -- 172.21.15.87:0/1399860268 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998009ca0 con 0x7f99a0153880 2024-01-20T08:32:01.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.488+0000 7f9995ffb640 1 -- 172.21.15.87:0/1399860268 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800f040 con 0x7f99a0153880 2024-01-20T08:32:01.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.488+0000 7f9995ffb640 1 -- 172.21.15.87:0/1399860268 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980146c0 con 0x7f99a0153880 2024-01-20T08:32:01.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.488+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a00c6100 con 0x7f99a0153880 2024-01-20T08:32:01.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.488+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a014a1d0 con 0x7f99a0153880 2024-01-20T08:32:01.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.491+0000 7f99aa628740 1 Processor -- start 2024-01-20T08:32:01.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.491+0000 7f99aa628740 1 -- start start 2024-01-20T08:32:01.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99aa628740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd861590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd860290 0x55eefd8a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99aa628740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd81c010 con 0x55eefd8a17f0 2024-01-20T08:32:01.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd7cde60 con 0x55eefd8a1bd0 2024-01-20T08:32:01.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eefd7cb550 con 0x55eefd860290 2024-01-20T08:32:01.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd861590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.492+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.493+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42122/0 (socket says 172.21.15.87:42122) 2024-01-20T08:32:01.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.493+0000 7f99a4ffc640 1 -- 172.21.15.87:0/670887748 learned_addr learned my addr 172.21.15.87:0/670887748 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.495+0000 7f99a4ffc640 1 -- 172.21.15.87:0/670887748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd860290 msgr2=0x55eefd8a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.495+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd860290 0x55eefd8a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.495+0000 7f99a4ffc640 1 -- 172.21.15.87:0/670887748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 msgr2=0x55eefd8a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.495+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.495+0000 7f99a4ffc640 1 -- 172.21.15.87:0/670887748 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eefd7beb00 con 0x55eefd8a17f0 2024-01-20T08:32:01.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.497+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd861590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99900097a0 tx=0x7f9990030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375d2e782fb7d46c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.499+0000 7f9982ffd640 1 -- 172.21.15.87:0/670887748 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999003e070 con 0x55eefd8a17f0 2024-01-20T08:32:01.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.501+0000 7f9982ffd640 1 -- 172.21.15.87:0/670887748 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990030bb0 con 0x55eefd8a17f0 2024-01-20T08:32:01.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.501+0000 7f9982ffd640 1 -- 172.21.15.87:0/670887748 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990039450 con 0x55eefd8a17f0 2024-01-20T08:32:01.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 -- 172.21.15.87:0/670887748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 msgr2=0x55eefd861590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd861590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99900097a0 tx=0x7f9990030000 comp rx=0 tx=0).stop 2024-01-20T08:32:01.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 -- 172.21.15.87:0/670887748 shutdown_connections 2024-01-20T08:32:01.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd860290 0x55eefd8a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 --2- 172.21.15.87:0/670887748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd861590 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 -- 172.21.15.87:0/670887748 >> 172.21.15.87:0/670887748 conn(0x55eefd85d400 msgr2=0x55eefd860ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 -- 172.21.15.87:0/670887748 shutdown_connections 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.502+0000 7f99aa628740 1 -- 172.21.15.87:0/670887748 wait complete. 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.503+0000 7f99aa628740 1 -- start start 2024-01-20T08:32:01.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.503+0000 7f99aa628740 1 Processor -- start 2024-01-20T08:32:01.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.504+0000 7f99aa628740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd87cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a1bd0 0x55eefd8b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 0x55eefd87d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99aa628740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd7c6e40 con 0x55eefd8a17f0 2024-01-20T08:32:01.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd780250 con 0x55eefd89c790 2024-01-20T08:32:01.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eefd77fef0 con 0x55eefd8a1bd0 2024-01-20T08:32:01.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99aa628740 10 client.0 _ll_register_callbacks cb 0x55eefd784720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:01.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd87cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 0x55eefd87d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 0x55eefd87d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:42126/0 (socket says 172.21.15.87:42126) 2024-01-20T08:32:01.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.505+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1015823367 learned_addr learned my addr 172.21.15.87:0/1015823367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.508+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a1bd0 msgr2=0x55eefd8b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.508+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a1bd0 0x55eefd8b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.508+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 msgr2=0x55eefd87cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.508+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd87cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.508+0000 7f99a4ffc640 1 -- 172.21.15.87:0/1015823367 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a015a130 con 0x55eefd89c790 2024-01-20T08:32:01.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.509+0000 7f9995ffb640 1 -- 172.21.15.87:0/1399860268 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9998014820 con 0x7f99a0153880 2024-01-20T08:32:01.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.510+0000 7f9995ffb640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f998406ae90 0x7f998406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.510+0000 7f99a57fd640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f998406ae90 0x7f998406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.510+0000 7f99a57fd640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f998406ae90 0x7f998406d350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55eefd765cd0 tx=0x7f999c009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.510+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 0x55eefd87d5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f99900098c0 tx=0x7f9990008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48182df0b680a991 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.510+0000 7f997bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-01-20T08:32:01.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.512+0000 7f9995ffb640 1 -- 172.21.15.87:0/1399860268 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9998058ec0 con 0x7f99a0153880 2024-01-20T08:32:01.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.514+0000 7f997bfff640 1 -- 172.21.15.87:0/1015823367 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990003810 con 0x55eefd89c790 2024-01-20T08:32:01.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.514+0000 7f997bfff640 1 -- 172.21.15.87:0/1015823367 <== mon.1 v2:172.21.15.116:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9990003970 con 0x55eefd89c790 2024-01-20T08:32:01.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.514+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eefd7bfcd0 con 0x55eefd89c790 2024-01-20T08:32:01.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.516+0000 7f997bfff640 1 -- 172.21.15.87:0/1015823367 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99900044f0 con 0x55eefd89c790 2024-01-20T08:32:01.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.516+0000 7f99aa628740 10 client.4813 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:01.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.516+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55eefd7c54c0 con 0x55eefd89c790 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.519+0000 7f997bfff640 1 -- 172.21.15.87:0/1015823367 <== mon.1 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eefd7bfcd0 con 0x55eefd89c790 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.519+0000 7f997bfff640 1 -- 172.21.15.87:0/1015823367 <== mon.1 v2:172.21.15.116:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f9990050070 con 0x55eefd89c790 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99aa628740 10 client.4813 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99aa628740 10 client.4813 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99aa628740 20 client.4813 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99aa628740 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd8b5140 0x55eefd8bff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99aa628740 4 client.4813 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-20T08:32:01.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 --> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eefd7bf9f0 con 0x55eefd8b5140 2024-01-20T08:32:01.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.520+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd8b5140 0x55eefd8bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.523+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd8b5140 0x55eefd8bff90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f307ab232cfb0e server_cookie=aa505109d4b20522 in_seq=0 out_seq=0 2024-01-20T08:32:01.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.523+0000 7f997bfff640 10 client.4813 ms_handle_connect on v2:172.21.15.87:6838/3149977016 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f997bfff640 1 -- 172.21.15.87:0/1015823367 <== mds.0 v2:172.21.15.87:6838/3149977016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eefd7bf9f0 con 0x55eefd8b5140 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f997bfff640 10 client.4813 handle_command_reply: tid=0 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 client.4813 shutdown 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 msgr2=0x55eefd87d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 0x55eefd87d5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f99900098c0 tx=0x7f9990008030 comp rx=0 tx=0).stop 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 shutdown_connections 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd8b5140 0x55eefd8bff90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd89c790 0x55eefd87d5c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a1bd0 0x55eefd8b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f997bfff640 0 client.4813 ms_handle_reset on v2:172.21.15.87:6838/3149977016 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 --2- 172.21.15.87:0/1015823367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a17f0 0x55eefd87cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.540+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 >> 172.21.15.87:0/1015823367 conn(0x55eefd860290 msgr2=0x55eefd861310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.541+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 shutdown_connections 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.541+0000 7f99aa628740 1 -- 172.21.15.87:0/1015823367 wait complete. 2024-01-20T08:32:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.541+0000 7f99aa628740 20 client.4813 trim_cache size 0 max 16384 2024-01-20T08:32:01.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.556+0000 7f99aa628740 1 Processor -- start 2024-01-20T08:32:01.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.556+0000 7f99aa628740 1 -- start start 2024-01-20T08:32:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.557+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd89c790 0x55eefd85f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.557+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a17f0 0x55eefd8a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.557+0000 7f99aa628740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.557+0000 7f99aa628740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd745300 con 0x55eefd8a1bd0 2024-01-20T08:32:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.557+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd8aac20 con 0x55eefd8a17f0 2024-01-20T08:32:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.557+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eefd87a030 con 0x55eefd89c790 2024-01-20T08:32:01.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.558+0000 7f99a5ffe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.558+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a17f0 0x55eefd8a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.558+0000 7f99a57fd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd89c790 0x55eefd85f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.558+0000 7f99a5ffe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34504/0 (socket says 172.21.15.87:34504) 2024-01-20T08:32:01.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.558+0000 7f99a5ffe640 1 -- 172.21.15.87:0/3733861353 learned_addr learned my addr 172.21.15.87:0/3733861353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.560+0000 7f99a5ffe640 1 -- 172.21.15.87:0/3733861353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd89c790 msgr2=0x55eefd85f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.560+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd89c790 0x55eefd85f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.560+0000 7f99a5ffe640 1 -- 172.21.15.87:0/3733861353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a17f0 msgr2=0x55eefd8a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.560+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a17f0 0x55eefd8a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.560+0000 7f99a5ffe640 1 -- 172.21.15.87:0/3733861353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eefd762150 con 0x55eefd8a1bd0 2024-01-20T08:32:01.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.562+0000 7f99a5ffe640 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9998059080 tx=0x7f999805d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46349ea138638c26 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.565+0000 7f99827fc640 1 -- 172.21.15.87:0/3733861353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999805f070 con 0x55eefd8a1bd0 2024-01-20T08:32:01.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.565+0000 7f99827fc640 1 -- 172.21.15.87:0/3733861353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999805b040 con 0x55eefd8a1bd0 2024-01-20T08:32:01.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.565+0000 7f99827fc640 1 -- 172.21.15.87:0/3733861353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998060ec0 con 0x55eefd8a1bd0 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 -- 172.21.15.87:0/3733861353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 msgr2=0x55eefd8a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9998059080 tx=0x7f999805d0e0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 -- 172.21.15.87:0/3733861353 shutdown_connections 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd8a1bd0 0x55eefd8a0f10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a17f0 0x55eefd8a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 --2- 172.21.15.87:0/3733861353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd89c790 0x55eefd85f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 -- 172.21.15.87:0/3733861353 >> 172.21.15.87:0/3733861353 conn(0x55eefd860290 msgr2=0x55eefd85b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 -- 172.21.15.87:0/3733861353 shutdown_connections 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.566+0000 7f99aa628740 1 -- 172.21.15.87:0/3733861353 wait complete. 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.567+0000 7f99aa628740 1 -- start start 2024-01-20T08:32:01.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.567+0000 7f99aa628740 1 Processor -- start 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 0x55eefd885b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a17f0 0x55eefd860ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8c4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd861220 con 0x55eefd89c790 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefd8c4ca0 con 0x55eefd8a1bd0 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eefd8a0ff0 con 0x55eefd8a17f0 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99aa628740 10 client.0 _ll_register_callbacks cb 0x55eefd8a1550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99a57fd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 0x55eefd885b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99a5ffe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.568+0000 7f99a4ffc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a17f0 0x55eefd860ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.569+0000 7f99a57fd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 0x55eefd885b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34510/0 (socket says 172.21.15.87:34510) 2024-01-20T08:32:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.569+0000 7f99a57fd640 1 -- 172.21.15.87:0/2906595834 learned_addr learned my addr 172.21.15.87:0/2906595834 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.571+0000 7f99a57fd640 1 -- 172.21.15.87:0/2906595834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a17f0 msgr2=0x55eefd860ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.571+0000 7f99a57fd640 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a17f0 0x55eefd860ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.571+0000 7f99a57fd640 1 -- 172.21.15.87:0/2906595834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 msgr2=0x55eefd8c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.571+0000 7f99a57fd640 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.571+0000 7f99a57fd640 1 -- 172.21.15.87:0/2906595834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a014a1d0 con 0x55eefd89c790 2024-01-20T08:32:01.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.573+0000 7f99a57fd640 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 0x55eefd885b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f999c00fea0 tx=0x7f999c007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369fbcc187c44fc3 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.573+0000 7f997bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-01-20T08:32:01.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.577+0000 7f997bfff640 1 -- 172.21.15.87:0/2906595834 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c014ea0 con 0x55eefd89c790 2024-01-20T08:32:01.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.577+0000 7f997bfff640 1 -- 172.21.15.87:0/2906595834 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f999c01c040 con 0x55eefd89c790 2024-01-20T08:32:01.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.577+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9990030bb0 con 0x55eefd89c790 2024-01-20T08:32:01.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.578+0000 7f997bfff640 1 -- 172.21.15.87:0/2906595834 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c021610 con 0x55eefd89c790 2024-01-20T08:32:01.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.578+0000 7f99aa628740 10 client.4785 fetch_fsmap learned FSMap version 18 2024-01-20T08:32:01.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.578+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55eefd787940 con 0x55eefd89c790 2024-01-20T08:32:01.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f997bfff640 1 -- 172.21.15.87:0/2906595834 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990030bb0 con 0x55eefd89c790 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f997bfff640 1 -- 172.21.15.87:0/2906595834 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 18) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f999c012070 con 0x55eefd89c790 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f99aa628740 10 client.4785 fetch_fsmap finished waiting for FSMap version 18 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f99aa628740 10 client.4785 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f99aa628740 20 client.4785 populate_metadata read hostname 'smithi087' 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f99aa628740 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd908cf0 0x55eefd9090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f99aa628740 4 client.4785 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.582+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 --> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eefd4c66c0 con 0x55eefd908cf0 2024-01-20T08:32:01.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.583+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd908cf0 0x55eefd9090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.586+0000 7f99a4ffc640 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd908cf0 0x55eefd9090d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25de109f94ee539 server_cookie=9b0c215106907eff in_seq=0 out_seq=0 2024-01-20T08:32:01.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.586+0000 7f997bfff640 10 client.4785 ms_handle_connect on v2:172.21.15.87:6838/3149977016 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f997bfff640 1 -- 172.21.15.87:0/2906595834 <== mds.0 v2:172.21.15.87:6838/3149977016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55eefd4c66c0 con 0x55eefd908cf0 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f997bfff640 10 client.4785 handle_command_reply: tid=0 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 client.4785 shutdown 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 msgr2=0x55eefd885b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 0x55eefd885b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f999c00fea0 tx=0x7f999c007b70 comp rx=0 tx=0).stop 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 shutdown_connections 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:6838/3149977016,v1:172.21.15.87:6839/3149977016] conn(0x55eefd908cf0 0x55eefd9090d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55eefd8a1bd0 0x55eefd8c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55eefd8a17f0 0x55eefd860ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f997bfff640 0 client.4785 ms_handle_reset on v2:172.21.15.87:6838/3149977016 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 --2- 172.21.15.87:0/2906595834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eefd89c790 0x55eefd885b60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.589+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 >> 172.21.15.87:0/2906595834 conn(0x55eefd860290 msgr2=0x55eefd85f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 shutdown_connections 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99aa628740 1 -- 172.21.15.87:0/2906595834 wait complete. 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99aa628740 20 client.4785 trim_cache size 0 max 16384 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "2b2378c9-6207-4be4-af26-3b0cbbc1772f", 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-01-20T08:32:01.594 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4293, 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 50.977007624000002, 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 18, 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 27, 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 27, 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 61.270251424000001 2024-01-20T08:32:01.595 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:32:01.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f998406ae90 msgr2=0x7f998406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f998406ae90 0x7f998406d350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55eefd765cd0 tx=0x7f999c009040 comp rx=0 tx=0).stop 2024-01-20T08:32:01.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 msgr2=0x7f99a014eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 0x7f99a014eb20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f999800efc0 tx=0x7f999800c490 comp rx=0 tx=0).stop 2024-01-20T08:32:01.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 shutdown_connections 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f998406ae90 0x7f998406d350 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f99a0153880 0x7f99a014eb20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99a00b1220 0x7f99a01efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 --2- 172.21.15.87:0/1399860268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99a00b0c60 0x7f99a00c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 >> 172.21.15.87:0/1399860268 conn(0x7f99a0072390 msgr2=0x7f99a00c1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 shutdown_connections 2024-01-20T08:32:01.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.590+0000 7f99a7a61640 1 -- 172.21.15.87:0/1399860268 wait complete. 2024-01-20T08:32:01.604 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-01-20T08:32:01.614 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-01-20T08:32:01.614 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-01-20T08:32:01.615 INFO:tasks.ceph_fuse:client.0 config is {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']} 2024-01-20T08:32:01.615 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-20T08:32:01.615 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:01.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:01.615 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-01-20T08:32:01.630 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-01-20T08:32:01.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:01.630 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-20T08:32:01.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:01.698 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-01-20T08:32:01.790 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-01-20T08:32:01.791 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-20T08:32:01.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.887+0000 7f8bd38ee640 1 Processor -- start 2024-01-20T08:32:01.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.887+0000 7f8bd38ee640 1 -- start start 2024-01-20T08:32:01.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.888+0000 7f8bd38ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 0x7f8bcc0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.888+0000 7f8bd38ee640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8bcc1573e0 0x7f8bcc0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.888+0000 7f8bd38ee640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc06b4f0 con 0x7f8bcc151690 2024-01-20T08:32:01.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.888+0000 7f8bd38ee640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c8670 con 0x7f8bcc1573e0 2024-01-20T08:32:01.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.888+0000 7f8bd38ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc157890 con 0x7f8bcc0c69b0 2024-01-20T08:32:01.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.889+0000 7f8bd0e89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 0x7f8bcc0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.889+0000 7f8bd0e89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 0x7f8bcc0cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49314/0 (socket says 172.21.15.87:49314) 2024-01-20T08:32:01.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.889+0000 7f8bd0e89640 1 -- 172.21.15.87:0/1106359120 learned_addr learned my addr 172.21.15.87:0/1106359120 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.889+0000 7f8bd1e8b640 1 --2- 172.21.15.87:0/1106359120 >> v2:172.21.15.116:3300/0 conn(0x7f8bcc1573e0 0x7f8bcc0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.891+0000 7f8bd0e89640 1 -- 172.21.15.87:0/1106359120 >> v2:172.21.15.116:3300/0 conn(0x7f8bcc1573e0 msgr2=0x7f8bcc0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:32:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.891+0000 7f8bd0e89640 1 --2- 172.21.15.87:0/1106359120 >> v2:172.21.15.116:3300/0 conn(0x7f8bcc1573e0 0x7f8bcc0cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.891+0000 7f8bd0e89640 1 -- 172.21.15.87:0/1106359120 >> v1:172.21.15.116:6789/0 conn(0x7f8bcc151690 legacy=0x7f8bcc06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.891+0000 7f8bd0e89640 1 -- 172.21.15.87:0/1106359120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc157b70 con 0x7f8bcc0c69b0 2024-01-20T08:32:01.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.893+0000 7f8bd0e89640 1 --2- 172.21.15.87:0/1106359120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 0x7f8bcc0cbe20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009870 tx=0x7f8bbc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6342e36bebb76da2 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.897+0000 7f8bc3fff640 1 -- 172.21.15.87:0/1106359120 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc030840 con 0x7f8bcc0c69b0 2024-01-20T08:32:01.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.897+0000 7f8bc3fff640 1 -- 172.21.15.87:0/1106359120 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc0309a0 con 0x7f8bcc0c69b0 2024-01-20T08:32:01.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.897+0000 7f8bc3fff640 1 -- 172.21.15.87:0/1106359120 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc030d30 con 0x7f8bcc0c69b0 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 -- 172.21.15.87:0/1106359120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 msgr2=0x7f8bcc0cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/1106359120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 0x7f8bcc0cbe20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009870 tx=0x7f8bbc02b980 comp rx=0 tx=0).stop 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 -- 172.21.15.87:0/1106359120 shutdown_connections 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/1106359120 >> v2:172.21.15.116:3300/0 conn(0x7f8bcc1573e0 0x7f8bcc0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/1106359120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc0c69b0 0x7f8bcc0cbe20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 -- 172.21.15.87:0/1106359120 >> 172.21.15.87:0/1106359120 conn(0x7f8bcc06ded0 msgr2=0x7f8bcc06e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 -- 172.21.15.87:0/1106359120 shutdown_connections 2024-01-20T08:32:01.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.898+0000 7f8bd38ee640 1 -- 172.21.15.87:0/1106359120 wait complete. 2024-01-20T08:32:01.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.899+0000 7f8bd38ee640 1 Processor -- start 2024-01-20T08:32:01.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.899+0000 7f8bd38ee640 1 -- start start 2024-01-20T08:32:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd38ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bcc0c69b0 0x7f8bcc1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd38ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc151690 0x7f8bcc0c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd38ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc1573e0 0x7f8bcc1f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd38ee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc1f3d20 con 0x7f8bcc1573e0 2024-01-20T08:32:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd38ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c1db0 con 0x7f8bcc0c69b0 2024-01-20T08:32:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd38ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c17f0 con 0x7f8bcc151690 2024-01-20T08:32:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.900+0000 7f8bd1e8b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc1573e0 0x7f8bcc1f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.901+0000 7f8bd0e89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc151690 0x7f8bcc0c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.901+0000 7f8bd168a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bcc0c69b0 0x7f8bcc1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.901+0000 7f8bd0e89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc151690 0x7f8bcc0c11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:49330/0 (socket says 172.21.15.87:49330) 2024-01-20T08:32:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.901+0000 7f8bd0e89640 1 -- 172.21.15.87:0/3859847385 learned_addr learned my addr 172.21.15.87:0/3859847385 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:32:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.904+0000 7f8bd1e8b640 1 -- 172.21.15.87:0/3859847385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc151690 msgr2=0x7f8bcc0c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.904+0000 7f8bd1e8b640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc151690 0x7f8bcc0c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.904+0000 7f8bd1e8b640 1 -- 172.21.15.87:0/3859847385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bcc0c69b0 msgr2=0x7f8bcc1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.904+0000 7f8bd1e8b640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bcc0c69b0 0x7f8bcc1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.904+0000 7f8bd1e8b640 1 -- 172.21.15.87:0/3859847385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0094d0 con 0x7f8bcc1573e0 2024-01-20T08:32:01.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.906+0000 7f8bd1e8b640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc1573e0 0x7f8bcc1f6b70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e7c0 tx=0x7f8bc400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.910+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400c730 con 0x7f8bcc1573e0 2024-01-20T08:32:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.910+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc400c890 con 0x7f8bcc1573e0 2024-01-20T08:32:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.910+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400cc80 con 0x7f8bcc1573e0 2024-01-20T08:32:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.910+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc0c1ad0 con 0x7f8bcc1573e0 2024-01-20T08:32:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.910+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1f39b0 con 0x7f8bcc1573e0 2024-01-20T08:32:01.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.911+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc066e40 con 0x7f8bcc1573e0 2024-01-20T08:32:01.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.929+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8bc4004020 con 0x7f8bcc1573e0 2024-01-20T08:32:01.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.929+0000 7f8bc1ffb640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8bb006af50 0x7f8bb006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:32:01.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.930+0000 7f8bd168a640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8bb006af50 0x7f8bb006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:32:01.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.930+0000 7f8bd168a640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8bb006af50 0x7f8bb006d410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8001f10 tx=0x7f8bc8009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:32:01.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:01.932+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f8bc4058110 con 0x7f8bcc1573e0 2024-01-20T08:32:02.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.164+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8bc4054450 con 0x7f8bcc1573e0 2024-01-20T08:32:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.376+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8bcc067890 con 0x7f8bcc1573e0 2024-01-20T08:32:02.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.424+0000 7f8bc1ffb640 1 -- 172.21.15.87:0/3859847385 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8bc4021070 con 0x7f8bcc1573e0 2024-01-20T08:32:02.427 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8bb006af50 msgr2=0x7f8bb006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8bb006af50 0x7f8bb006d410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8001f10 tx=0x7f8bc8009040 comp rx=0 tx=0).stop 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc1573e0 msgr2=0x7f8bcc1f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc1573e0 0x7f8bcc1f6b70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e7c0 tx=0x7f8bc400ec80 comp rx=0 tx=0).stop 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 shutdown_connections 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8bb006af50 0x7f8bb006d410 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc1573e0 0x7f8bcc1f6b70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bcc151690 0x7f8bcc0c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 --2- 172.21.15.87:0/3859847385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bcc0c69b0 0x7f8bcc1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 >> 172.21.15.87:0/3859847385 conn(0x7f8bcc06ded0 msgr2=0x7f8bcc14d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.426+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 shutdown_connections 2024-01-20T08:32:02.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:02.427+0000 7f8bd38ee640 1 -- 172.21.15.87:0/3859847385 wait complete. 2024-01-20T08:32:02.440 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-20T08:32:02.440 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-20T08:32:02.440 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-01-20T08:32:02.441 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-20T08:32:02.441 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:02.441 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-20T08:32:02.441 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-20T08:32:02.441 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-20T08:32:02.441 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-20T08:32:02.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.441 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-01-20T08:32:02.456 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-20T08:32:02.456 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-20T08:32:02.456 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-01-20T08:32:02.456 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-01-20T08:32:02.456 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-01-20T08:32:02.456 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-20T08:32:02.457 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42350sec preferred_lft 42350sec 2024-01-20T08:32:02.458 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-01-20T08:32:02.458 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-01-20T08:32:02.458 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-20T08:32:02.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-20T08:32:02.458 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:32:02.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.566 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-20T08:32:02.591 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-01-20T08:32:02.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.593 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-01-20T08:32:02.649 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-01-20T08:32:02.650 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-20T08:32:02.650 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-20T08:32:02.650 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-20T08:32:02.650 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-20T08:32:02.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.650 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:32:02.650 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:32:02.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-20T08:32:02.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-20T08:32:02.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-20T08:32:02.650 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:32:02.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.745 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-01-20T08:32:02.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.760 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-01-20T08:32:02.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.815 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:32:02.815 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:32:02.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:02.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-20T08:32:02.815 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:32:02.896 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-20T08:32:02.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:02.896 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:32:02.896 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:32:02.896 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-20T08:32:02.896 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-20T08:32:02.896 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-20T08:32:02.897 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-20T08:32:02.897 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-20T08:32:02.897 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:32:03.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:03.021 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-01-20T08:32:03.021 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-01-20T08:32:03.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-01-20T08:32:03.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-01-20T08:32:03.021 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-01-20T08:32:03.062 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']} 2024-01-20T08:32:03.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-20T08:32:03.062 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:03.117 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-20T08:32:03.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-20T08:32:03.118 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:03.172 DEBUG:tasks.ceph_manager:running client.0 under valgrind with args ['sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/client.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-20T08:32:03.173 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:03.241 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:03.297 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:03.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:03.299 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:03.299 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:03.299 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:03.299 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:03.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:03.299 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:03.351 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-20T08:32:03.351 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-20T08:32:03.394 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:03.420 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:03.477 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:03.478 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:03.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:03.479 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:04.633 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:04.661 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:04.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:04.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:04.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:04.718 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:05.871 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:05.899 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:05.955 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:05.956 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:05.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:05.957 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:07.111 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:07.138 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:07.194 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:07.194 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:07.194 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:07.194 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:07.194 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:07.195 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:07.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:07.196 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:08.350 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:08.378 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:08.433 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:08.434 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:08.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:08.435 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:09.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[47020]: starting ceph client 2024-01-20T08:32:09.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-01-20T08:32:08.905+0000 64f6100 -1 init, newargv = 0x8203970 newargc=15 2024-01-20T08:32:09.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:09.602 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:32:09.616 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:09.657 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:09.672 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:09.673 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:09.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:09.674 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:10.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[47020]: starting fuse 2024-01-20T08:32:10.830 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-01-20T08:32:10.858 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-01-20T08:32:10.915 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-01-20T08:32:10.916 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:10.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:10.917 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-01-20T08:32:10.970 INFO:teuthology.orchestra.run.smithi087.stdout:41 2024-01-20T08:32:10.970 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:32:10.970 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-01-20T08:32:10.971 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-01-20T08:32:10.972 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-20T08:32:10.972 DEBUG:teuthology.orchestra.run.smithi087:> 2024-01-20T08:32:10.972 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-01-20T08:32:10.972 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-01-20T08:32:11.089 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.47020.asok 2024-01-20T08:32:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:32:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-20T08:32:11.099 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47020.asok 2024-01-20T08:32:11.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:11.100 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47020.asok status 2024-01-20T08:32:11.331 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:32:11.331 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-01-20T08:32:11.331 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "a00c9940991c5e1e9ab595faa3f6e1281d8ae167", 2024-01-20T08:32:11.331 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-763-ga00c9940 (a00c9940991c5e1e9ab595faa3f6e1281d8ae167) squid (dev)", 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "47020", 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4652, 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4652 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-01-20T08:32:11.332 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 832347037 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 832347037 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4652 172.21.15.87:0/832347037", 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "172.21.15.87:0/832347037", 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 18, 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 27, 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-01-20T08:32:11.333 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-01-20T08:32:11.334 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-01-20T08:32:11.334 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:32:11.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:11.341 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-01-20T08:32:11.341 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:11.399 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-01-20T08:32:11.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:11.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:32:11.400 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:11.450 INFO:teuthology.run_tasks:Running task workunit... 2024-01-20T08:32:11.462 INFO:tasks.workunit:Pulling workunits from ref a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:32:11.463 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-01-20T08:32:11.463 INFO:tasks.workunit:timeout=3h 2024-01-20T08:32:11.463 INFO:tasks.workunit:cleanup=True 2024-01-20T08:32:11.463 DEBUG:teuthology.orchestra.run.smithi087:> stat -- /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:11.478 INFO:teuthology.orchestra.run.smithi087.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:11.478 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-01-20T08:32:11.478 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-01-20T08:32:11.478 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-20T08:32:11.479 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:fusefs_t:s0 2024-01-20T08:32:11.479 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-01-20T08:32:11.479 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-01-20 08:31:10.899290042 +0000 2024-01-20T08:32:11.479 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-01-20 08:31:25.429137078 +0000 2024-01-20T08:32:11.479 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-01-20T08:32:11.479 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-01-20T08:32:11.479 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-01-20T08:32:12.138 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a00c9940991c5e1e9ab595faa3f6e1281d8ae167 2024-01-20T08:32:12.153 INFO:tasks.workunit.client.0.smithi087.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-01-20T08:32:39.716 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:32:39.719 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr:Note: switching to 'a00c9940991c5e1e9ab595faa3f6e1281d8ae167'. 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr:changes and commit them, and you can discard any commits you make in this 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr:state without impacting any branches by switching back to a branch. 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.545 INFO:tasks.workunit.client.0.smithi087.stderr:If you want to create a new branch to retain commits you create, you may 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr:do so (now or later) by using -c with the switch command. Example: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: git switch -c 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr:Or undo this operation with: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: git switch - 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-01-20T08:32:59.546 INFO:tasks.workunit.client.0.smithi087.stderr:HEAD is now at a00c9940991 Merge pull request #54990 from ifed01/wip-ifed-fix-expand-with-ncb 2024-01-20T08:32:59.549 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-01-20T08:32:59.568 INFO:tasks.workunit.client.0.smithi087.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-01-20T08:32:59.571 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-20T08:32:59.571 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-01-20T08:33:00.216 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-01-20T08:33:00.281 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-01-20T08:33:00.320 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-20T08:33:00.322 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-20T08:33:00.322 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-01-20T08:33:00.360 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-20T08:33:00.363 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:33:00.363 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-01-20T08:33:00.419 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-01-20T08:33:00.420 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-01-20T08:33:00.421 DEBUG:teuthology.orchestra.run.smithi087:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a00c9940991c5e1e9ab595faa3f6e1281d8ae167 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-01-20T08:33:00.847 INFO:tasks.workunit.client.0.smithi087.stderr:+ dbench 1 2024-01-20T08:33:00.859 INFO:tasks.workunit.client.0.smithi087.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-20T08:33:00.859 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-01-20T08:33:00.859 INFO:tasks.workunit.client.0.smithi087.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-20T08:33:00.868 INFO:tasks.workunit.client.0.smithi087.stdout:0 of 1 processes prepared for launch 0 sec 2024-01-20T08:33:00.968 INFO:tasks.workunit.client.0.smithi087.stdout:1 of 1 processes prepared for launch 0 sec 2024-01-20T08:33:00.968 INFO:tasks.workunit.client.0.smithi087.stdout:releasing clients 2024-01-20T08:33:01.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 31 1.05 MB/sec warmup 1 sec latency 52.377 ms 2024-01-20T08:33:02.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 111 2.62 MB/sec warmup 2 sec latency 208.398 ms 2024-01-20T08:33:03.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 155 2.18 MB/sec warmup 3 sec latency 175.210 ms 2024-01-20T08:33:04.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 197 1.96 MB/sec warmup 4 sec latency 180.171 ms 2024-01-20T08:33:05.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 247 1.88 MB/sec warmup 5 sec latency 162.760 ms 2024-01-20T08:33:06.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 287 1.78 MB/sec warmup 6 sec latency 207.196 ms 2024-01-20T08:33:07.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 323 1.68 MB/sec warmup 7 sec latency 203.002 ms 2024-01-20T08:33:08.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 365 1.63 MB/sec warmup 8 sec latency 255.503 ms 2024-01-20T08:33:09.798 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:33:09.801 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:33:09.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 405 1.59 MB/sec warmup 9 sec latency 263.792 ms 2024-01-20T08:33:10.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 445 1.56 MB/sec warmup 10 sec latency 234.550 ms 2024-01-20T08:33:11.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 477 1.52 MB/sec warmup 11 sec latency 273.905 ms 2024-01-20T08:33:12.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 511 1.48 MB/sec warmup 12 sec latency 207.549 ms 2024-01-20T08:33:13.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 606 1.61 MB/sec warmup 13 sec latency 294.369 ms 2024-01-20T08:33:14.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 639 1.50 MB/sec warmup 14 sec latency 383.199 ms 2024-01-20T08:33:15.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 652 1.40 MB/sec warmup 15 sec latency 311.067 ms 2024-01-20T08:33:16.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671 1.32 MB/sec warmup 16 sec latency 460.824 ms 2024-01-20T08:33:17.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 699 1.25 MB/sec warmup 17 sec latency 293.608 ms 2024-01-20T08:33:18.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 726 1.18 MB/sec warmup 18 sec latency 289.528 ms 2024-01-20T08:33:19.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 757 1.12 MB/sec warmup 19 sec latency 313.847 ms 2024-01-20T08:33:20.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 796 1.07 MB/sec warmup 20 sec latency 207.687 ms 2024-01-20T08:33:21.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 813 1.02 MB/sec warmup 21 sec latency 495.761 ms 2024-01-20T08:33:22.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 853 0.98 MB/sec warmup 22 sec latency 311.038 ms 2024-01-20T08:33:23.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 862 0.94 MB/sec warmup 23 sec latency 310.694 ms 2024-01-20T08:33:24.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 891 0.90 MB/sec warmup 24 sec latency 326.297 ms 2024-01-20T08:33:25.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 926 0.87 MB/sec warmup 25 sec latency 229.450 ms 2024-01-20T08:33:26.969 INFO:tasks.workunit.client.0.smithi087.stdout: 1 930 0.84 MB/sec warmup 26 sec latency 351.263 ms 2024-01-20T08:33:27.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 942 0.81 MB/sec warmup 27 sec latency 308.489 ms 2024-01-20T08:33:28.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 983 0.78 MB/sec warmup 28 sec latency 279.411 ms 2024-01-20T08:33:29.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1027 0.79 MB/sec warmup 29 sec latency 318.634 ms 2024-01-20T08:33:30.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1046 0.76 MB/sec warmup 30 sec latency 299.632 ms 2024-01-20T08:33:31.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1050 0.74 MB/sec warmup 31 sec latency 474.686 ms 2024-01-20T08:33:32.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1088 0.72 MB/sec warmup 32 sec latency 132.520 ms 2024-01-20T08:33:33.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1107 0.70 MB/sec warmup 33 sec latency 310.803 ms 2024-01-20T08:33:34.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1139 0.68 MB/sec warmup 34 sec latency 306.454 ms 2024-01-20T08:33:35.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1168 0.66 MB/sec warmup 35 sec latency 286.593 ms 2024-01-20T08:33:36.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1173 0.64 MB/sec warmup 36 sec latency 323.855 ms 2024-01-20T08:33:37.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1204 0.62 MB/sec warmup 37 sec latency 148.766 ms 2024-01-20T08:33:38.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1231 0.61 MB/sec warmup 38 sec latency 300.561 ms 2024-01-20T08:33:39.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:33:39.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:33:39.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1252 0.59 MB/sec warmup 39 sec latency 314.211 ms 2024-01-20T08:33:40.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1280 0.58 MB/sec warmup 40 sec latency 334.217 ms 2024-01-20T08:33:41.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1285 0.56 MB/sec warmup 41 sec latency 537.558 ms 2024-01-20T08:33:42.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1381 0.62 MB/sec warmup 42 sec latency 292.460 ms 2024-01-20T08:33:43.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1397 0.60 MB/sec warmup 43 sec latency 310.533 ms 2024-01-20T08:33:44.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1418 0.59 MB/sec warmup 44 sec latency 303.728 ms 2024-01-20T08:33:45.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1448 0.58 MB/sec warmup 45 sec latency 221.931 ms 2024-01-20T08:33:46.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1479 0.57 MB/sec warmup 46 sec latency 384.865 ms 2024-01-20T08:33:47.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1490 0.55 MB/sec warmup 47 sec latency 303.331 ms 2024-01-20T08:33:48.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1525 0.54 MB/sec warmup 48 sec latency 312.307 ms 2024-01-20T08:33:49.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1552 0.53 MB/sec warmup 49 sec latency 294.097 ms 2024-01-20T08:33:50.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1578 0.52 MB/sec warmup 50 sec latency 296.996 ms 2024-01-20T08:33:51.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1603 0.51 MB/sec warmup 51 sec latency 305.042 ms 2024-01-20T08:33:52.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1615 0.50 MB/sec warmup 52 sec latency 305.991 ms 2024-01-20T08:33:53.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1641 0.49 MB/sec warmup 53 sec latency 328.992 ms 2024-01-20T08:33:54.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1663 0.49 MB/sec warmup 54 sec latency 303.218 ms 2024-01-20T08:33:55.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1701 0.48 MB/sec warmup 55 sec latency 298.936 ms 2024-01-20T08:33:56.970 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1747 0.47 MB/sec warmup 56 sec latency 304.091 ms 2024-01-20T08:33:57.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1789 0.46 MB/sec warmup 57 sec latency 294.414 ms 2024-01-20T08:33:58.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1820 0.45 MB/sec warmup 58 sec latency 321.832 ms 2024-01-20T08:33:59.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1839 0.45 MB/sec warmup 59 sec latency 306.402 ms 2024-01-20T08:34:00.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1858 0.44 MB/sec warmup 60 sec latency 316.797 ms 2024-01-20T08:34:01.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1869 0.43 MB/sec warmup 61 sec latency 415.515 ms 2024-01-20T08:34:02.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1903 0.43 MB/sec warmup 62 sec latency 304.418 ms 2024-01-20T08:34:03.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1920 0.42 MB/sec warmup 63 sec latency 350.961 ms 2024-01-20T08:34:04.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 1975 0.41 MB/sec warmup 64 sec latency 296.491 ms 2024-01-20T08:34:05.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2020 0.41 MB/sec warmup 65 sec latency 306.288 ms 2024-01-20T08:34:06.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2043 0.40 MB/sec warmup 66 sec latency 312.933 ms 2024-01-20T08:34:07.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2060 0.40 MB/sec warmup 67 sec latency 324.267 ms 2024-01-20T08:34:08.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2081 0.39 MB/sec warmup 68 sec latency 330.667 ms 2024-01-20T08:34:09.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:34:09.920 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:34:09.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2096 0.39 MB/sec warmup 69 sec latency 325.113 ms 2024-01-20T08:34:10.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2140 0.38 MB/sec warmup 70 sec latency 366.902 ms 2024-01-20T08:34:11.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2173 0.38 MB/sec warmup 71 sec latency 472.164 ms 2024-01-20T08:34:12.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2183 0.37 MB/sec warmup 72 sec latency 315.288 ms 2024-01-20T08:34:13.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2221 0.37 MB/sec warmup 73 sec latency 339.116 ms 2024-01-20T08:34:14.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2292 0.38 MB/sec warmup 74 sec latency 302.463 ms 2024-01-20T08:34:15.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2306 0.37 MB/sec warmup 75 sec latency 312.055 ms 2024-01-20T08:34:16.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2324 0.37 MB/sec warmup 76 sec latency 436.662 ms 2024-01-20T08:34:17.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2337 0.36 MB/sec warmup 77 sec latency 300.902 ms 2024-01-20T08:34:18.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2361 0.36 MB/sec warmup 78 sec latency 293.118 ms 2024-01-20T08:34:19.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2366 0.35 MB/sec warmup 79 sec latency 304.781 ms 2024-01-20T08:34:20.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2383 0.35 MB/sec warmup 80 sec latency 324.066 ms 2024-01-20T08:34:21.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2399 0.35 MB/sec warmup 81 sec latency 362.951 ms 2024-01-20T08:34:22.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2426 0.34 MB/sec warmup 82 sec latency 307.798 ms 2024-01-20T08:34:23.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2434 0.34 MB/sec warmup 83 sec latency 302.467 ms 2024-01-20T08:34:24.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2453 0.33 MB/sec warmup 84 sec latency 163.928 ms 2024-01-20T08:34:25.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2489 0.34 MB/sec warmup 85 sec latency 315.795 ms 2024-01-20T08:34:26.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2501 0.33 MB/sec warmup 86 sec latency 323.302 ms 2024-01-20T08:34:27.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2551 0.33 MB/sec warmup 87 sec latency 298.022 ms 2024-01-20T08:34:28.971 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2567 0.33 MB/sec warmup 88 sec latency 320.490 ms 2024-01-20T08:34:29.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2582 0.33 MB/sec warmup 89 sec latency 340.450 ms 2024-01-20T08:34:30.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2600 0.32 MB/sec warmup 90 sec latency 262.021 ms 2024-01-20T08:34:31.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2615 0.32 MB/sec warmup 91 sec latency 303.920 ms 2024-01-20T08:34:32.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2622 0.32 MB/sec warmup 92 sec latency 318.304 ms 2024-01-20T08:34:33.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2750 0.34 MB/sec warmup 93 sec latency 286.479 ms 2024-01-20T08:34:34.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2768 0.34 MB/sec warmup 94 sec latency 343.953 ms 2024-01-20T08:34:35.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2872 0.34 MB/sec warmup 95 sec latency 288.211 ms 2024-01-20T08:34:36.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2897 0.34 MB/sec warmup 96 sec latency 311.967 ms 2024-01-20T08:34:37.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2931 0.35 MB/sec warmup 97 sec latency 168.587 ms 2024-01-20T08:34:38.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2967 0.34 MB/sec warmup 98 sec latency 300.675 ms 2024-01-20T08:34:39.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 2982 0.34 MB/sec warmup 99 sec latency 326.261 ms 2024-01-20T08:34:40.060 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:34:40.063 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:34:40.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3000 0.34 MB/sec warmup 100 sec latency 290.613 ms 2024-01-20T08:34:41.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3024 0.33 MB/sec warmup 101 sec latency 323.482 ms 2024-01-20T08:34:42.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3171 0.33 MB/sec warmup 102 sec latency 353.259 ms 2024-01-20T08:34:43.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3190 0.33 MB/sec warmup 103 sec latency 319.064 ms 2024-01-20T08:34:44.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3266 0.34 MB/sec warmup 104 sec latency 241.565 ms 2024-01-20T08:34:45.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3280 0.34 MB/sec warmup 105 sec latency 310.629 ms 2024-01-20T08:34:46.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3283 0.33 MB/sec warmup 106 sec latency 330.159 ms 2024-01-20T08:34:47.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3287 0.33 MB/sec warmup 107 sec latency 367.589 ms 2024-01-20T08:34:48.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3292 0.33 MB/sec warmup 108 sec latency 326.662 ms 2024-01-20T08:34:49.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3295 0.32 MB/sec warmup 109 sec latency 386.869 ms 2024-01-20T08:34:50.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3303 0.32 MB/sec warmup 110 sec latency 414.560 ms 2024-01-20T08:34:51.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3307 0.32 MB/sec warmup 111 sec latency 416.732 ms 2024-01-20T08:34:52.972 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3311 0.32 MB/sec warmup 112 sec latency 300.773 ms 2024-01-20T08:34:53.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3314 0.31 MB/sec warmup 113 sec latency 323.347 ms 2024-01-20T08:34:54.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3318 0.31 MB/sec warmup 114 sec latency 300.803 ms 2024-01-20T08:34:55.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3323 0.31 MB/sec warmup 115 sec latency 301.934 ms 2024-01-20T08:34:56.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3326 0.30 MB/sec warmup 116 sec latency 307.590 ms 2024-01-20T08:34:57.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3331 0.30 MB/sec warmup 117 sec latency 314.306 ms 2024-01-20T08:34:58.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3335 0.30 MB/sec warmup 118 sec latency 302.764 ms 2024-01-20T08:34:59.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3341 0.30 MB/sec warmup 119 sec latency 315.060 ms 2024-01-20T08:35:01.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3391 0.05 MB/sec execute 1 sec latency 172.303 ms 2024-01-20T08:35:02.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3398 0.02 MB/sec execute 2 sec latency 325.429 ms 2024-01-20T08:35:03.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3519 0.97 MB/sec execute 3 sec latency 304.563 ms 2024-01-20T08:35:04.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3548 0.74 MB/sec execute 4 sec latency 305.296 ms 2024-01-20T08:35:05.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3563 0.60 MB/sec execute 5 sec latency 349.941 ms 2024-01-20T08:35:06.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3588 0.50 MB/sec execute 6 sec latency 387.441 ms 2024-01-20T08:35:07.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3633 0.45 MB/sec execute 7 sec latency 307.422 ms 2024-01-20T08:35:08.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3663 0.41 MB/sec execute 8 sec latency 302.855 ms 2024-01-20T08:35:09.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3675 0.36 MB/sec execute 9 sec latency 400.279 ms 2024-01-20T08:35:10.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:35:10.171 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:35:10.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3696 0.33 MB/sec execute 10 sec latency 323.015 ms 2024-01-20T08:35:11.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3723 0.30 MB/sec execute 11 sec latency 132.822 ms 2024-01-20T08:35:12.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3770 0.36 MB/sec execute 12 sec latency 295.789 ms 2024-01-20T08:35:13.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3785 0.33 MB/sec execute 13 sec latency 182.101 ms 2024-01-20T08:35:14.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3796 0.31 MB/sec execute 14 sec latency 211.563 ms 2024-01-20T08:35:15.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3808 0.29 MB/sec execute 15 sec latency 206.275 ms 2024-01-20T08:35:16.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3824 0.27 MB/sec execute 16 sec latency 186.655 ms 2024-01-20T08:35:17.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3840 0.25 MB/sec execute 17 sec latency 172.756 ms 2024-01-20T08:35:18.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3856 0.24 MB/sec execute 18 sec latency 179.426 ms 2024-01-20T08:35:19.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3870 0.23 MB/sec execute 19 sec latency 176.238 ms 2024-01-20T08:35:20.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3883 0.22 MB/sec execute 20 sec latency 330.171 ms 2024-01-20T08:35:21.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3898 0.22 MB/sec execute 21 sec latency 328.728 ms 2024-01-20T08:35:22.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3921 0.21 MB/sec execute 22 sec latency 282.629 ms 2024-01-20T08:35:23.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3937 0.20 MB/sec execute 23 sec latency 188.039 ms 2024-01-20T08:35:24.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3947 0.19 MB/sec execute 24 sec latency 203.573 ms 2024-01-20T08:35:25.973 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3962 0.18 MB/sec execute 25 sec latency 174.308 ms 2024-01-20T08:35:26.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3976 0.18 MB/sec execute 26 sec latency 152.489 ms 2024-01-20T08:35:27.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3993 0.17 MB/sec execute 27 sec latency 163.699 ms 2024-01-20T08:35:28.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4010 0.17 MB/sec execute 28 sec latency 147.710 ms 2024-01-20T08:35:29.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4027 0.16 MB/sec execute 29 sec latency 153.110 ms 2024-01-20T08:35:30.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4035 0.15 MB/sec execute 30 sec latency 334.363 ms 2024-01-20T08:35:31.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4047 0.15 MB/sec execute 31 sec latency 282.122 ms 2024-01-20T08:35:32.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4067 0.14 MB/sec execute 32 sec latency 298.845 ms 2024-01-20T08:35:33.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4090 0.14 MB/sec execute 33 sec latency 306.276 ms 2024-01-20T08:35:34.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4112 0.14 MB/sec execute 34 sec latency 232.456 ms 2024-01-20T08:35:35.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4138 0.13 MB/sec execute 35 sec latency 242.820 ms 2024-01-20T08:35:36.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4153 0.13 MB/sec execute 36 sec latency 403.824 ms 2024-01-20T08:35:37.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4172 0.13 MB/sec execute 37 sec latency 243.586 ms 2024-01-20T08:35:38.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4181 0.12 MB/sec execute 38 sec latency 313.369 ms 2024-01-20T08:35:39.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4217 0.12 MB/sec execute 39 sec latency 349.011 ms 2024-01-20T08:35:40.317 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:35:40.321 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:35:40.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4226 0.12 MB/sec execute 40 sec latency 347.520 ms 2024-01-20T08:35:41.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4254 0.12 MB/sec execute 41 sec latency 303.551 ms 2024-01-20T08:35:42.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4292 0.12 MB/sec execute 42 sec latency 292.623 ms 2024-01-20T08:35:43.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4318 0.12 MB/sec execute 43 sec latency 301.562 ms 2024-01-20T08:35:44.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4365 0.12 MB/sec execute 44 sec latency 215.955 ms 2024-01-20T08:35:45.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4386 0.11 MB/sec execute 45 sec latency 314.266 ms 2024-01-20T08:35:46.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4421 0.11 MB/sec execute 46 sec latency 432.262 ms 2024-01-20T08:35:47.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4432 0.11 MB/sec execute 47 sec latency 367.616 ms 2024-01-20T08:35:48.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4450 0.11 MB/sec execute 48 sec latency 347.990 ms 2024-01-20T08:35:49.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4488 0.11 MB/sec execute 49 sec latency 288.473 ms 2024-01-20T08:35:50.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4500 0.11 MB/sec execute 50 sec latency 326.785 ms 2024-01-20T08:35:51.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4511 0.11 MB/sec execute 51 sec latency 368.602 ms 2024-01-20T08:35:52.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4540 0.11 MB/sec execute 52 sec latency 284.741 ms 2024-01-20T08:35:53.974 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4596 0.12 MB/sec execute 53 sec latency 322.588 ms 2024-01-20T08:35:54.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4614 0.12 MB/sec execute 54 sec latency 319.261 ms 2024-01-20T08:35:55.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4619 0.12 MB/sec execute 55 sec latency 321.242 ms 2024-01-20T08:35:56.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4654 0.12 MB/sec execute 56 sec latency 157.220 ms 2024-01-20T08:35:57.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4680 0.12 MB/sec execute 57 sec latency 327.053 ms 2024-01-20T08:35:58.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4711 0.12 MB/sec execute 58 sec latency 313.831 ms 2024-01-20T08:35:59.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4734 0.11 MB/sec execute 59 sec latency 288.899 ms 2024-01-20T08:36:00.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4739 0.11 MB/sec execute 60 sec latency 316.306 ms 2024-01-20T08:36:01.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4770 0.11 MB/sec execute 61 sec latency 184.410 ms 2024-01-20T08:36:02.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4799 0.11 MB/sec execute 62 sec latency 297.475 ms 2024-01-20T08:36:03.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4821 0.11 MB/sec execute 63 sec latency 333.113 ms 2024-01-20T08:36:04.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4847 0.11 MB/sec execute 64 sec latency 354.028 ms 2024-01-20T08:36:05.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4942 0.15 MB/sec execute 65 sec latency 339.327 ms 2024-01-20T08:36:06.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4947 0.15 MB/sec execute 66 sec latency 384.588 ms 2024-01-20T08:36:07.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4963 0.14 MB/sec execute 67 sec latency 299.741 ms 2024-01-20T08:36:08.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 4985 0.14 MB/sec execute 68 sec latency 336.995 ms 2024-01-20T08:36:09.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5014 0.14 MB/sec execute 69 sec latency 323.934 ms 2024-01-20T08:36:10.405 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:36:10.408 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:36:10.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5047 0.14 MB/sec execute 70 sec latency 412.113 ms 2024-01-20T08:36:11.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5056 0.14 MB/sec execute 71 sec latency 282.257 ms 2024-01-20T08:36:12.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5091 0.14 MB/sec execute 72 sec latency 299.716 ms 2024-01-20T08:36:13.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5118 0.13 MB/sec execute 73 sec latency 307.992 ms 2024-01-20T08:36:14.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5141 0.13 MB/sec execute 74 sec latency 301.116 ms 2024-01-20T08:36:15.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5168 0.13 MB/sec execute 75 sec latency 305.955 ms 2024-01-20T08:36:16.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5180 0.13 MB/sec execute 76 sec latency 344.605 ms 2024-01-20T08:36:17.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5208 0.13 MB/sec execute 77 sec latency 310.348 ms 2024-01-20T08:36:18.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5245 0.13 MB/sec execute 78 sec latency 302.597 ms 2024-01-20T08:36:19.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5266 0.13 MB/sec execute 79 sec latency 229.741 ms 2024-01-20T08:36:20.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5323 0.12 MB/sec execute 80 sec latency 301.558 ms 2024-01-20T08:36:21.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5358 0.12 MB/sec execute 81 sec latency 297.628 ms 2024-01-20T08:36:22.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5386 0.12 MB/sec execute 82 sec latency 320.414 ms 2024-01-20T08:36:23.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5402 0.12 MB/sec execute 83 sec latency 304.944 ms 2024-01-20T08:36:24.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5423 0.12 MB/sec execute 84 sec latency 305.064 ms 2024-01-20T08:36:25.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5434 0.12 MB/sec execute 85 sec latency 309.154 ms 2024-01-20T08:36:26.975 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5468 0.12 MB/sec execute 86 sec latency 341.423 ms 2024-01-20T08:36:27.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5485 0.12 MB/sec execute 87 sec latency 294.129 ms 2024-01-20T08:36:28.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5540 0.12 MB/sec execute 88 sec latency 305.325 ms 2024-01-20T08:36:29.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5573 0.11 MB/sec execute 89 sec latency 299.573 ms 2024-01-20T08:36:30.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5605 0.11 MB/sec execute 90 sec latency 296.393 ms 2024-01-20T08:36:31.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5621 0.11 MB/sec execute 91 sec latency 300.187 ms 2024-01-20T08:36:32.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5643 0.11 MB/sec execute 92 sec latency 329.343 ms 2024-01-20T08:36:33.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5658 0.11 MB/sec execute 93 sec latency 324.472 ms 2024-01-20T08:36:34.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5702 0.11 MB/sec execute 94 sec latency 344.683 ms 2024-01-20T08:36:35.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5735 0.11 MB/sec execute 95 sec latency 344.398 ms 2024-01-20T08:36:36.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5744 0.11 MB/sec execute 96 sec latency 409.936 ms 2024-01-20T08:36:37.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5782 0.11 MB/sec execute 97 sec latency 306.313 ms 2024-01-20T08:36:38.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5853 0.12 MB/sec execute 98 sec latency 320.658 ms 2024-01-20T08:36:39.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5866 0.12 MB/sec execute 99 sec latency 295.996 ms 2024-01-20T08:36:40.510 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:36:40.513 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:36:40.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5884 0.12 MB/sec execute 100 sec latency 444.156 ms 2024-01-20T08:36:41.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5897 0.11 MB/sec execute 101 sec latency 302.479 ms 2024-01-20T08:36:42.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5921 0.11 MB/sec execute 102 sec latency 293.596 ms 2024-01-20T08:36:43.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5926 0.11 MB/sec execute 103 sec latency 315.466 ms 2024-01-20T08:36:44.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5943 0.11 MB/sec execute 104 sec latency 549.202 ms 2024-01-20T08:36:45.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5959 0.11 MB/sec execute 105 sec latency 294.907 ms 2024-01-20T08:36:46.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5976 0.11 MB/sec execute 106 sec latency 384.918 ms 2024-01-20T08:36:47.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5992 0.11 MB/sec execute 107 sec latency 301.970 ms 2024-01-20T08:36:48.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6009 0.11 MB/sec execute 108 sec latency 348.411 ms 2024-01-20T08:36:49.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6043 0.11 MB/sec execute 109 sec latency 325.871 ms 2024-01-20T08:36:50.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6054 0.11 MB/sec execute 110 sec latency 310.570 ms 2024-01-20T08:36:51.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6107 0.11 MB/sec execute 111 sec latency 310.268 ms 2024-01-20T08:36:52.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6118 0.11 MB/sec execute 112 sec latency 308.503 ms 2024-01-20T08:36:53.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6139 0.11 MB/sec execute 113 sec latency 304.069 ms 2024-01-20T08:36:54.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6158 0.11 MB/sec execute 114 sec latency 344.279 ms 2024-01-20T08:36:55.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6173 0.11 MB/sec execute 115 sec latency 325.405 ms 2024-01-20T08:36:56.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6179 0.11 MB/sec execute 116 sec latency 467.992 ms 2024-01-20T08:36:57.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6302 0.13 MB/sec execute 117 sec latency 301.747 ms 2024-01-20T08:36:58.976 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6317 0.13 MB/sec execute 118 sec latency 317.679 ms 2024-01-20T08:36:59.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6336 0.13 MB/sec execute 119 sec latency 294.149 ms 2024-01-20T08:37:00.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6435 0.13 MB/sec execute 120 sec latency 302.288 ms 2024-01-20T08:37:01.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6477 0.14 MB/sec execute 121 sec latency 150.051 ms 2024-01-20T08:37:02.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6521 0.14 MB/sec execute 122 sec latency 295.625 ms 2024-01-20T08:37:03.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6530 0.14 MB/sec execute 123 sec latency 321.295 ms 2024-01-20T08:37:04.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6549 0.14 MB/sec execute 124 sec latency 204.006 ms 2024-01-20T08:37:05.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6574 0.14 MB/sec execute 125 sec latency 304.921 ms 2024-01-20T08:37:06.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6722 0.14 MB/sec execute 126 sec latency 443.475 ms 2024-01-20T08:37:07.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6743 0.14 MB/sec execute 127 sec latency 337.049 ms 2024-01-20T08:37:08.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6792 0.14 MB/sec execute 128 sec latency 315.517 ms 2024-01-20T08:37:09.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6829 0.15 MB/sec execute 129 sec latency 320.239 ms 2024-01-20T08:37:10.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:37:10.602 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:37:10.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6837 0.15 MB/sec execute 130 sec latency 312.832 ms 2024-01-20T08:37:11.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6841 0.15 MB/sec execute 131 sec latency 310.114 ms 2024-01-20T08:37:12.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6846 0.14 MB/sec execute 132 sec latency 359.582 ms 2024-01-20T08:37:13.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6849 0.14 MB/sec execute 133 sec latency 389.864 ms 2024-01-20T08:37:14.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6854 0.14 MB/sec execute 134 sec latency 404.680 ms 2024-01-20T08:37:15.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6862 0.14 MB/sec execute 135 sec latency 396.129 ms 2024-01-20T08:37:16.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6865 0.14 MB/sec execute 136 sec latency 433.839 ms 2024-01-20T08:37:17.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6868 0.14 MB/sec execute 137 sec latency 305.708 ms 2024-01-20T08:37:18.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6872 0.14 MB/sec execute 138 sec latency 302.682 ms 2024-01-20T08:37:19.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6877 0.14 MB/sec execute 139 sec latency 298.237 ms 2024-01-20T08:37:20.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6880 0.14 MB/sec execute 140 sec latency 309.342 ms 2024-01-20T08:37:21.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6885 0.14 MB/sec execute 141 sec latency 310.317 ms 2024-01-20T08:37:22.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6889 0.13 MB/sec execute 142 sec latency 306.083 ms 2024-01-20T08:37:23.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6892 0.13 MB/sec execute 143 sec latency 300.032 ms 2024-01-20T08:37:24.977 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6919 0.13 MB/sec execute 144 sec latency 282.941 ms 2024-01-20T08:37:25.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6945 0.13 MB/sec execute 145 sec latency 136.604 ms 2024-01-20T08:37:26.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 6949 0.13 MB/sec execute 146 sec latency 385.034 ms 2024-01-20T08:37:27.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7062 0.15 MB/sec execute 147 sec latency 305.761 ms 2024-01-20T08:37:28.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7085 0.15 MB/sec execute 148 sec latency 298.554 ms 2024-01-20T08:37:29.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7109 0.15 MB/sec execute 149 sec latency 297.820 ms 2024-01-20T08:37:30.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7130 0.15 MB/sec execute 150 sec latency 296.536 ms 2024-01-20T08:37:31.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7174 0.15 MB/sec execute 151 sec latency 491.613 ms 2024-01-20T08:37:32.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7191 0.15 MB/sec execute 152 sec latency 314.220 ms 2024-01-20T08:37:33.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7218 0.15 MB/sec execute 153 sec latency 328.751 ms 2024-01-20T08:37:34.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7234 0.15 MB/sec execute 154 sec latency 334.998 ms 2024-01-20T08:37:35.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7261 0.14 MB/sec execute 155 sec latency 152.746 ms 2024-01-20T08:37:36.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7312 0.15 MB/sec execute 156 sec latency 297.044 ms 2024-01-20T08:37:37.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7326 0.15 MB/sec execute 157 sec latency 190.446 ms 2024-01-20T08:37:38.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7336 0.15 MB/sec execute 158 sec latency 195.517 ms 2024-01-20T08:37:39.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7350 0.15 MB/sec execute 159 sec latency 224.081 ms 2024-01-20T08:37:40.721 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:37:40.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:37:40.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7367 0.15 MB/sec execute 160 sec latency 203.088 ms 2024-01-20T08:37:41.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7381 0.15 MB/sec execute 161 sec latency 234.315 ms 2024-01-20T08:37:42.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7397 0.14 MB/sec execute 162 sec latency 257.889 ms 2024-01-20T08:37:43.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7414 0.14 MB/sec execute 163 sec latency 145.770 ms 2024-01-20T08:37:44.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7424 0.14 MB/sec execute 164 sec latency 315.158 ms 2024-01-20T08:37:45.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7442 0.14 MB/sec execute 165 sec latency 307.649 ms 2024-01-20T08:37:46.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7463 0.14 MB/sec execute 166 sec latency 299.224 ms 2024-01-20T08:37:47.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7479 0.14 MB/sec execute 167 sec latency 170.908 ms 2024-01-20T08:37:48.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7488 0.14 MB/sec execute 168 sec latency 212.438 ms 2024-01-20T08:37:49.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7503 0.14 MB/sec execute 169 sec latency 182.301 ms 2024-01-20T08:37:50.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7519 0.14 MB/sec execute 170 sec latency 149.922 ms 2024-01-20T08:37:51.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7535 0.14 MB/sec execute 171 sec latency 162.177 ms 2024-01-20T08:37:52.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7553 0.14 MB/sec execute 172 sec latency 165.292 ms 2024-01-20T08:37:53.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7570 0.14 MB/sec execute 173 sec latency 160.543 ms 2024-01-20T08:37:54.978 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7578 0.14 MB/sec execute 174 sec latency 321.584 ms 2024-01-20T08:37:55.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7591 0.14 MB/sec execute 175 sec latency 301.987 ms 2024-01-20T08:37:56.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7609 0.13 MB/sec execute 176 sec latency 307.371 ms 2024-01-20T08:37:57.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7627 0.13 MB/sec execute 177 sec latency 308.806 ms 2024-01-20T08:37:58.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7654 0.13 MB/sec execute 178 sec latency 173.272 ms 2024-01-20T08:37:59.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7678 0.13 MB/sec execute 179 sec latency 246.050 ms 2024-01-20T08:38:00.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7699 0.13 MB/sec execute 180 sec latency 365.295 ms 2024-01-20T08:38:01.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7715 0.13 MB/sec execute 181 sec latency 194.470 ms 2024-01-20T08:38:02.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7723 0.13 MB/sec execute 182 sec latency 330.721 ms 2024-01-20T08:38:03.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7759 0.13 MB/sec execute 183 sec latency 356.772 ms 2024-01-20T08:38:04.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7775 0.13 MB/sec execute 184 sec latency 333.841 ms 2024-01-20T08:38:05.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7808 0.13 MB/sec execute 185 sec latency 300.490 ms 2024-01-20T08:38:06.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7846 0.13 MB/sec execute 186 sec latency 228.164 ms 2024-01-20T08:38:07.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7882 0.13 MB/sec execute 187 sec latency 297.287 ms 2024-01-20T08:38:08.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7919 0.13 MB/sec execute 188 sec latency 255.886 ms 2024-01-20T08:38:09.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7929 0.13 MB/sec execute 189 sec latency 334.812 ms 2024-01-20T08:38:10.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:38:10.855 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:38:10.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7965 0.13 MB/sec execute 190 sec latency 309.165 ms 2024-01-20T08:38:11.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 7974 0.13 MB/sec execute 191 sec latency 337.552 ms 2024-01-20T08:38:12.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8004 0.13 MB/sec execute 192 sec latency 359.340 ms 2024-01-20T08:38:13.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8039 0.13 MB/sec execute 193 sec latency 233.929 ms 2024-01-20T08:38:14.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8043 0.13 MB/sec execute 194 sec latency 297.569 ms 2024-01-20T08:38:15.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8053 0.13 MB/sec execute 195 sec latency 303.939 ms 2024-01-20T08:38:16.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8101 0.13 MB/sec execute 196 sec latency 284.730 ms 2024-01-20T08:38:17.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8138 0.13 MB/sec execute 197 sec latency 318.651 ms 2024-01-20T08:38:18.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8156 0.13 MB/sec execute 198 sec latency 333.049 ms 2024-01-20T08:38:19.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8170 0.13 MB/sec execute 199 sec latency 310.594 ms 2024-01-20T08:38:20.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8200 0.13 MB/sec execute 200 sec latency 292.917 ms 2024-01-20T08:38:21.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8223 0.13 MB/sec execute 201 sec latency 370.429 ms 2024-01-20T08:38:22.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8256 0.13 MB/sec execute 202 sec latency 284.808 ms 2024-01-20T08:38:23.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8276 0.13 MB/sec execute 203 sec latency 298.222 ms 2024-01-20T08:38:24.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8285 0.13 MB/sec execute 204 sec latency 323.753 ms 2024-01-20T08:38:25.979 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8312 0.13 MB/sec execute 205 sec latency 272.934 ms 2024-01-20T08:38:26.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8352 0.13 MB/sec execute 206 sec latency 299.395 ms 2024-01-20T08:38:27.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8381 0.13 MB/sec execute 207 sec latency 374.590 ms 2024-01-20T08:38:28.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8390 0.12 MB/sec execute 208 sec latency 335.425 ms 2024-01-20T08:38:29.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8485 0.14 MB/sec execute 209 sec latency 312.079 ms 2024-01-20T08:38:30.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8494 0.14 MB/sec execute 210 sec latency 318.912 ms 2024-01-20T08:38:31.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8506 0.14 MB/sec execute 211 sec latency 295.204 ms 2024-01-20T08:38:32.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8528 0.14 MB/sec execute 212 sec latency 404.916 ms 2024-01-20T08:38:33.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8557 0.14 MB/sec execute 213 sec latency 244.845 ms 2024-01-20T08:38:34.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8590 0.13 MB/sec execute 214 sec latency 343.827 ms 2024-01-20T08:38:35.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8599 0.13 MB/sec execute 215 sec latency 293.802 ms 2024-01-20T08:38:36.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8633 0.13 MB/sec execute 216 sec latency 314.236 ms 2024-01-20T08:38:37.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8661 0.13 MB/sec execute 217 sec latency 308.635 ms 2024-01-20T08:38:38.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8682 0.13 MB/sec execute 218 sec latency 294.689 ms 2024-01-20T08:38:39.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8711 0.13 MB/sec execute 219 sec latency 308.315 ms 2024-01-20T08:38:40.934 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:38:40.937 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:38:40.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8723 0.13 MB/sec execute 220 sec latency 302.295 ms 2024-01-20T08:38:41.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8749 0.13 MB/sec execute 221 sec latency 314.574 ms 2024-01-20T08:38:42.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8770 0.13 MB/sec execute 222 sec latency 316.083 ms 2024-01-20T08:38:43.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8807 0.13 MB/sec execute 223 sec latency 255.931 ms 2024-01-20T08:38:44.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8844 0.13 MB/sec execute 224 sec latency 303.272 ms 2024-01-20T08:38:45.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8891 0.13 MB/sec execute 225 sec latency 309.276 ms 2024-01-20T08:38:46.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8928 0.13 MB/sec execute 226 sec latency 329.767 ms 2024-01-20T08:38:47.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8947 0.13 MB/sec execute 227 sec latency 299.177 ms 2024-01-20T08:38:48.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8967 0.13 MB/sec execute 228 sec latency 329.292 ms 2024-01-20T08:38:49.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8978 0.13 MB/sec execute 229 sec latency 298.124 ms 2024-01-20T08:38:50.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9012 0.13 MB/sec execute 230 sec latency 356.717 ms 2024-01-20T08:38:51.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9029 0.13 MB/sec execute 231 sec latency 446.158 ms 2024-01-20T08:38:52.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9083 0.13 MB/sec execute 232 sec latency 297.721 ms 2024-01-20T08:38:53.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9109 0.13 MB/sec execute 233 sec latency 319.323 ms 2024-01-20T08:38:54.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9149 0.13 MB/sec execute 234 sec latency 286.159 ms 2024-01-20T08:38:55.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9164 0.13 MB/sec execute 235 sec latency 297.948 ms 2024-01-20T08:38:56.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9187 0.12 MB/sec execute 236 sec latency 325.247 ms 2024-01-20T08:38:57.980 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9202 0.12 MB/sec execute 237 sec latency 304.769 ms 2024-01-20T08:38:58.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9246 0.12 MB/sec execute 238 sec latency 359.543 ms 2024-01-20T08:38:59.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9279 0.12 MB/sec execute 239 sec latency 333.590 ms 2024-01-20T08:39:00.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9288 0.12 MB/sec execute 240 sec latency 327.578 ms 2024-01-20T08:39:01.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9326 0.12 MB/sec execute 241 sec latency 324.929 ms 2024-01-20T08:39:02.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9397 0.13 MB/sec execute 242 sec latency 315.526 ms 2024-01-20T08:39:03.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9407 0.13 MB/sec execute 243 sec latency 424.001 ms 2024-01-20T08:39:04.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9428 0.13 MB/sec execute 244 sec latency 434.739 ms 2024-01-20T08:39:05.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9439 0.12 MB/sec execute 245 sec latency 294.102 ms 2024-01-20T08:39:06.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9465 0.12 MB/sec execute 246 sec latency 297.196 ms 2024-01-20T08:39:07.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9468 0.12 MB/sec execute 247 sec latency 496.299 ms 2024-01-20T08:39:08.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9477 0.12 MB/sec execute 248 sec latency 539.501 ms 2024-01-20T08:39:09.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9500 0.12 MB/sec execute 249 sec latency 303.026 ms 2024-01-20T08:39:10.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9520 0.12 MB/sec execute 250 sec latency 340.369 ms 2024-01-20T08:39:11.062 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:39:11.065 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:39:11.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9536 0.12 MB/sec execute 251 sec latency 456.317 ms 2024-01-20T08:39:12.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9553 0.12 MB/sec execute 252 sec latency 304.186 ms 2024-01-20T08:39:13.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9581 0.12 MB/sec execute 253 sec latency 323.265 ms 2024-01-20T08:39:14.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9598 0.12 MB/sec execute 254 sec latency 308.094 ms 2024-01-20T08:39:15.717 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:09:00.194 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:15.783 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:09:00.261 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:15.820 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:09:00.254 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:15.821 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:09:00.256 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:15.880 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:09:00.315 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:15.882 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:09:00.317 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:15.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9651 0.12 MB/sec execute 255 sec latency 349.519 ms 2024-01-20T08:39:16.279 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:09:00.714 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:16.286 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:09:00.721 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:16.297 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:09:00.775 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:16.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9662 0.12 MB/sec execute 256 sec latency 388.864 ms 2024-01-20T08:39:17.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9682 0.12 MB/sec execute 257 sec latency 296.799 ms 2024-01-20T08:39:18.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9698 0.12 MB/sec execute 258 sec latency 338.956 ms 2024-01-20T08:39:19.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9711 0.12 MB/sec execute 259 sec latency 290.552 ms 2024-01-20T08:39:20.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9720 0.12 MB/sec execute 260 sec latency 335.897 ms 2024-01-20T08:39:21.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9819 0.13 MB/sec execute 261 sec latency 405.033 ms 2024-01-20T08:39:22.981 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9860 0.13 MB/sec execute 262 sec latency 411.378 ms 2024-01-20T08:39:23.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9880 0.13 MB/sec execute 263 sec latency 344.813 ms 2024-01-20T08:39:24.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 9979 0.13 MB/sec execute 264 sec latency 295.383 ms 2024-01-20T08:39:25.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10017 0.14 MB/sec execute 265 sec latency 149.380 ms 2024-01-20T08:39:26.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10042 0.14 MB/sec execute 266 sec latency 328.432 ms 2024-01-20T08:39:27.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10073 0.14 MB/sec execute 267 sec latency 369.713 ms 2024-01-20T08:39:28.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10093 0.14 MB/sec execute 268 sec latency 448.099 ms 2024-01-20T08:39:29.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10117 0.14 MB/sec execute 269 sec latency 300.316 ms 2024-01-20T08:39:30.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10266 0.14 MB/sec execute 270 sec latency 321.271 ms 2024-01-20T08:39:31.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10287 0.14 MB/sec execute 271 sec latency 376.245 ms 2024-01-20T08:39:32.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10331 0.14 MB/sec execute 272 sec latency 310.757 ms 2024-01-20T08:39:33.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10374 0.14 MB/sec execute 273 sec latency 298.760 ms 2024-01-20T08:39:34.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10381 0.14 MB/sec execute 274 sec latency 303.790 ms 2024-01-20T08:39:35.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10385 0.14 MB/sec execute 275 sec latency 349.763 ms 2024-01-20T08:39:36.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10390 0.14 MB/sec execute 276 sec latency 359.155 ms 2024-01-20T08:39:37.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10393 0.14 MB/sec execute 277 sec latency 366.905 ms 2024-01-20T08:39:38.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10398 0.14 MB/sec execute 278 sec latency 328.014 ms 2024-01-20T08:39:39.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10406 0.14 MB/sec execute 279 sec latency 367.147 ms 2024-01-20T08:39:40.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10410 0.14 MB/sec execute 280 sec latency 318.365 ms 2024-01-20T08:39:41.209 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:39:41.212 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:39:41.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10412 0.14 MB/sec execute 281 sec latency 391.429 ms 2024-01-20T08:39:42.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10417 0.14 MB/sec execute 282 sec latency 453.399 ms 2024-01-20T08:39:43.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10421 0.13 MB/sec execute 283 sec latency 301.763 ms 2024-01-20T08:39:44.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10424 0.13 MB/sec execute 284 sec latency 313.410 ms 2024-01-20T08:39:45.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10429 0.13 MB/sec execute 285 sec latency 308.759 ms 2024-01-20T08:39:46.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10433 0.13 MB/sec execute 286 sec latency 311.990 ms 2024-01-20T08:39:47.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10436 0.13 MB/sec execute 287 sec latency 307.712 ms 2024-01-20T08:39:48.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10463 0.13 MB/sec execute 288 sec latency 297.907 ms 2024-01-20T08:39:49.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10489 0.13 MB/sec execute 289 sec latency 139.805 ms 2024-01-20T08:39:50.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10496 0.13 MB/sec execute 290 sec latency 315.380 ms 2024-01-20T08:39:51.982 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10611 0.14 MB/sec execute 291 sec latency 314.002 ms 2024-01-20T08:39:52.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10636 0.14 MB/sec execute 292 sec latency 298.622 ms 2024-01-20T08:39:53.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10656 0.14 MB/sec execute 293 sec latency 293.036 ms 2024-01-20T08:39:54.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10679 0.14 MB/sec execute 294 sec latency 298.462 ms 2024-01-20T08:39:55.839 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:09:40.316 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.842 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:09:40.319 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.851 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:09:40.286 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.852 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:09:40.286 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.854 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:09:40.289 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.854 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:09:40.289 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.899 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:09:40.376 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.904 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:09:40.339 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.904 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:09:40.339 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:39:55.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10720 0.14 MB/sec execute 295 sec latency 411.699 ms 2024-01-20T08:39:56.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10736 0.14 MB/sec execute 296 sec latency 335.273 ms 2024-01-20T08:39:57.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10764 0.14 MB/sec execute 297 sec latency 316.985 ms 2024-01-20T08:39:58.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10785 0.14 MB/sec execute 298 sec latency 284.129 ms 2024-01-20T08:39:59.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10810 0.14 MB/sec execute 299 sec latency 142.961 ms 2024-01-20T08:40:00.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10858 0.14 MB/sec execute 300 sec latency 302.309 ms 2024-01-20T08:40:01.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10875 0.14 MB/sec execute 301 sec latency 179.059 ms 2024-01-20T08:40:02.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10885 0.14 MB/sec execute 302 sec latency 202.830 ms 2024-01-20T08:40:03.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10898 0.14 MB/sec execute 303 sec latency 216.158 ms 2024-01-20T08:40:04.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10914 0.14 MB/sec execute 304 sec latency 161.417 ms 2024-01-20T08:40:05.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10930 0.14 MB/sec execute 305 sec latency 202.752 ms 2024-01-20T08:40:06.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10945 0.14 MB/sec execute 306 sec latency 256.463 ms 2024-01-20T08:40:07.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10960 0.14 MB/sec execute 307 sec latency 171.924 ms 2024-01-20T08:40:08.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10973 0.14 MB/sec execute 308 sec latency 321.288 ms 2024-01-20T08:40:09.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10988 0.14 MB/sec execute 309 sec latency 318.411 ms 2024-01-20T08:40:10.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11009 0.14 MB/sec execute 310 sec latency 309.261 ms 2024-01-20T08:40:11.344 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:40:11.346 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:40:11.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11025 0.14 MB/sec execute 311 sec latency 177.459 ms 2024-01-20T08:40:12.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11034 0.14 MB/sec execute 312 sec latency 187.999 ms 2024-01-20T08:40:13.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11049 0.14 MB/sec execute 313 sec latency 177.089 ms 2024-01-20T08:40:14.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11066 0.14 MB/sec execute 314 sec latency 152.158 ms 2024-01-20T08:40:15.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11084 0.14 MB/sec execute 315 sec latency 151.164 ms 2024-01-20T08:40:16.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11099 0.14 MB/sec execute 316 sec latency 184.559 ms 2024-01-20T08:40:17.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11117 0.14 MB/sec execute 317 sec latency 146.468 ms 2024-01-20T08:40:18.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11125 0.13 MB/sec execute 318 sec latency 337.390 ms 2024-01-20T08:40:19.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11137 0.13 MB/sec execute 319 sec latency 268.590 ms 2024-01-20T08:40:20.983 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11157 0.13 MB/sec execute 320 sec latency 320.757 ms 2024-01-20T08:40:21.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11176 0.13 MB/sec execute 321 sec latency 314.762 ms 2024-01-20T08:40:22.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11202 0.13 MB/sec execute 322 sec latency 190.154 ms 2024-01-20T08:40:23.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11226 0.13 MB/sec execute 323 sec latency 226.606 ms 2024-01-20T08:40:24.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11244 0.13 MB/sec execute 324 sec latency 447.307 ms 2024-01-20T08:40:25.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11262 0.13 MB/sec execute 325 sec latency 244.958 ms 2024-01-20T08:40:26.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11270 0.13 MB/sec execute 326 sec latency 317.475 ms 2024-01-20T08:40:27.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11306 0.13 MB/sec execute 327 sec latency 419.456 ms 2024-01-20T08:40:28.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11316 0.13 MB/sec execute 328 sec latency 335.156 ms 2024-01-20T08:40:29.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11343 0.13 MB/sec execute 329 sec latency 292.065 ms 2024-01-20T08:40:30.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11381 0.13 MB/sec execute 330 sec latency 280.107 ms 2024-01-20T08:40:31.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11406 0.13 MB/sec execute 331 sec latency 313.915 ms 2024-01-20T08:40:32.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11451 0.13 MB/sec execute 332 sec latency 349.550 ms 2024-01-20T08:40:33.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11473 0.13 MB/sec execute 333 sec latency 303.237 ms 2024-01-20T08:40:34.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11512 0.13 MB/sec execute 334 sec latency 307.916 ms 2024-01-20T08:40:35.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11521 0.13 MB/sec execute 335 sec latency 297.325 ms 2024-01-20T08:40:36.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11544 0.13 MB/sec execute 336 sec latency 406.472 ms 2024-01-20T08:40:37.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11584 0.13 MB/sec execute 337 sec latency 221.001 ms 2024-01-20T08:40:38.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11589 0.13 MB/sec execute 338 sec latency 307.301 ms 2024-01-20T08:40:39.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11600 0.13 MB/sec execute 339 sec latency 300.082 ms 2024-01-20T08:40:40.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11638 0.13 MB/sec execute 340 sec latency 293.410 ms 2024-01-20T08:40:41.612 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:40:41.615 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:40:41.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11685 0.13 MB/sec execute 341 sec latency 325.767 ms 2024-01-20T08:40:42.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11703 0.13 MB/sec execute 342 sec latency 295.419 ms 2024-01-20T08:40:43.984 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11708 0.13 MB/sec execute 343 sec latency 309.725 ms 2024-01-20T08:40:44.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11745 0.13 MB/sec execute 344 sec latency 157.686 ms 2024-01-20T08:40:45.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11770 0.13 MB/sec execute 345 sec latency 287.444 ms 2024-01-20T08:40:46.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11803 0.13 MB/sec execute 346 sec latency 293.007 ms 2024-01-20T08:40:47.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11824 0.13 MB/sec execute 347 sec latency 304.855 ms 2024-01-20T08:40:48.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11832 0.13 MB/sec execute 348 sec latency 316.128 ms 2024-01-20T08:40:49.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11860 0.13 MB/sec execute 349 sec latency 205.621 ms 2024-01-20T08:40:50.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11898 0.13 MB/sec execute 350 sec latency 302.640 ms 2024-01-20T08:40:51.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11928 0.13 MB/sec execute 351 sec latency 351.978 ms 2024-01-20T08:40:52.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 11937 0.13 MB/sec execute 352 sec latency 334.997 ms 2024-01-20T08:40:53.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12033 0.14 MB/sec execute 353 sec latency 305.029 ms 2024-01-20T08:40:54.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12041 0.14 MB/sec execute 354 sec latency 297.214 ms 2024-01-20T08:40:55.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12059 0.14 MB/sec execute 355 sec latency 307.126 ms 2024-01-20T08:40:56.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12084 0.13 MB/sec execute 356 sec latency 362.692 ms 2024-01-20T08:40:57.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12117 0.13 MB/sec execute 357 sec latency 311.315 ms 2024-01-20T08:40:58.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12141 0.13 MB/sec execute 358 sec latency 305.107 ms 2024-01-20T08:40:59.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12156 0.13 MB/sec execute 359 sec latency 313.294 ms 2024-01-20T08:41:00.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12182 0.13 MB/sec execute 360 sec latency 230.407 ms 2024-01-20T08:41:01.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12210 0.13 MB/sec execute 361 sec latency 324.525 ms 2024-01-20T08:41:02.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12240 0.13 MB/sec execute 362 sec latency 302.499 ms 2024-01-20T08:41:03.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12261 0.13 MB/sec execute 363 sec latency 308.495 ms 2024-01-20T08:41:04.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12272 0.13 MB/sec execute 364 sec latency 305.435 ms 2024-01-20T08:41:05.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12299 0.13 MB/sec execute 365 sec latency 324.007 ms 2024-01-20T08:41:06.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12341 0.13 MB/sec execute 366 sec latency 323.132 ms 2024-01-20T08:41:07.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12356 0.13 MB/sec execute 367 sec latency 346.925 ms 2024-01-20T08:41:08.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12414 0.13 MB/sec execute 368 sec latency 297.732 ms 2024-01-20T08:41:09.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12457 0.13 MB/sec execute 369 sec latency 297.564 ms 2024-01-20T08:41:10.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12478 0.13 MB/sec execute 370 sec latency 325.213 ms 2024-01-20T08:41:11.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:41:11.670 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:41:11.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12494 0.13 MB/sec execute 371 sec latency 309.580 ms 2024-01-20T08:41:12.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12514 0.13 MB/sec execute 372 sec latency 302.133 ms 2024-01-20T08:41:13.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12527 0.13 MB/sec execute 373 sec latency 287.255 ms 2024-01-20T08:41:14.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12561 0.13 MB/sec execute 374 sec latency 333.777 ms 2024-01-20T08:41:15.985 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12578 0.13 MB/sec execute 375 sec latency 313.549 ms 2024-01-20T08:41:16.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12633 0.13 MB/sec execute 376 sec latency 288.621 ms 2024-01-20T08:41:17.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12660 0.13 MB/sec execute 377 sec latency 467.184 ms 2024-01-20T08:41:18.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12698 0.13 MB/sec execute 378 sec latency 294.324 ms 2024-01-20T08:41:19.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12713 0.13 MB/sec execute 379 sec latency 309.464 ms 2024-01-20T08:41:20.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12736 0.13 MB/sec execute 380 sec latency 307.288 ms 2024-01-20T08:41:21.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12751 0.13 MB/sec execute 381 sec latency 326.490 ms 2024-01-20T08:41:22.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12795 0.13 MB/sec execute 382 sec latency 348.306 ms 2024-01-20T08:41:23.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12828 0.13 MB/sec execute 383 sec latency 336.128 ms 2024-01-20T08:41:24.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12837 0.13 MB/sec execute 384 sec latency 332.632 ms 2024-01-20T08:41:25.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12875 0.13 MB/sec execute 385 sec latency 320.637 ms 2024-01-20T08:41:26.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12947 0.13 MB/sec execute 386 sec latency 289.173 ms 2024-01-20T08:41:27.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12957 0.13 MB/sec execute 387 sec latency 392.496 ms 2024-01-20T08:41:28.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12978 0.13 MB/sec execute 388 sec latency 424.186 ms 2024-01-20T08:41:29.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12989 0.13 MB/sec execute 389 sec latency 302.860 ms 2024-01-20T08:41:30.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13015 0.13 MB/sec execute 390 sec latency 313.317 ms 2024-01-20T08:41:31.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13018 0.13 MB/sec execute 391 sec latency 423.870 ms 2024-01-20T08:41:32.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13025 0.13 MB/sec execute 392 sec latency 665.326 ms 2024-01-20T08:41:33.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13048 0.13 MB/sec execute 393 sec latency 324.443 ms 2024-01-20T08:41:34.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13064 0.13 MB/sec execute 394 sec latency 312.275 ms 2024-01-20T08:41:35.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13086 0.13 MB/sec execute 395 sec latency 338.236 ms 2024-01-20T08:41:36.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13102 0.13 MB/sec execute 396 sec latency 302.144 ms 2024-01-20T08:41:37.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13128 0.13 MB/sec execute 397 sec latency 318.529 ms 2024-01-20T08:41:38.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13148 0.13 MB/sec execute 398 sec latency 313.491 ms 2024-01-20T08:41:39.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13201 0.13 MB/sec execute 399 sec latency 343.936 ms 2024-01-20T08:41:40.986 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13212 0.13 MB/sec execute 400 sec latency 312.655 ms 2024-01-20T08:41:41.798 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:41:41.800 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:41:41.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13232 0.13 MB/sec execute 401 sec latency 319.247 ms 2024-01-20T08:41:42.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13252 0.13 MB/sec execute 402 sec latency 370.447 ms 2024-01-20T08:41:43.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13267 0.13 MB/sec execute 403 sec latency 290.731 ms 2024-01-20T08:41:44.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13273 0.13 MB/sec execute 404 sec latency 352.974 ms 2024-01-20T08:41:45.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13372 0.13 MB/sec execute 405 sec latency 323.847 ms 2024-01-20T08:41:46.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13411 0.13 MB/sec execute 406 sec latency 343.707 ms 2024-01-20T08:41:47.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13431 0.13 MB/sec execute 407 sec latency 513.647 ms 2024-01-20T08:41:48.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13530 0.13 MB/sec execute 408 sec latency 303.197 ms 2024-01-20T08:41:49.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13568 0.14 MB/sec execute 409 sec latency 161.245 ms 2024-01-20T08:41:50.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13590 0.14 MB/sec execute 410 sec latency 320.280 ms 2024-01-20T08:41:51.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13624 0.13 MB/sec execute 411 sec latency 319.246 ms 2024-01-20T08:41:52.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13644 0.13 MB/sec execute 412 sec latency 482.870 ms 2024-01-20T08:41:53.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13667 0.13 MB/sec execute 413 sec latency 303.993 ms 2024-01-20T08:41:54.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13817 0.14 MB/sec execute 414 sec latency 356.020 ms 2024-01-20T08:41:55.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13838 0.14 MB/sec execute 415 sec latency 307.017 ms 2024-01-20T08:41:56.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13862 0.14 MB/sec execute 416 sec latency 344.010 ms 2024-01-20T08:41:57.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13920 0.14 MB/sec execute 417 sec latency 308.372 ms 2024-01-20T08:41:58.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13931 0.14 MB/sec execute 418 sec latency 307.744 ms 2024-01-20T08:41:59.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13935 0.14 MB/sec execute 419 sec latency 295.207 ms 2024-01-20T08:42:00.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13940 0.14 MB/sec execute 420 sec latency 366.777 ms 2024-01-20T08:42:01.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13943 0.14 MB/sec execute 421 sec latency 341.808 ms 2024-01-20T08:42:02.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13948 0.14 MB/sec execute 422 sec latency 454.677 ms 2024-01-20T08:42:03.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13955 0.14 MB/sec execute 423 sec latency 340.791 ms 2024-01-20T08:42:04.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13958 0.14 MB/sec execute 424 sec latency 357.463 ms 2024-01-20T08:42:05.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13961 0.13 MB/sec execute 425 sec latency 331.442 ms 2024-01-20T08:42:06.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13965 0.13 MB/sec execute 426 sec latency 338.941 ms 2024-01-20T08:42:07.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13969 0.13 MB/sec execute 427 sec latency 430.528 ms 2024-01-20T08:42:08.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13973 0.13 MB/sec execute 428 sec latency 296.177 ms 2024-01-20T08:42:09.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13976 0.13 MB/sec execute 429 sec latency 300.123 ms 2024-01-20T08:42:10.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13981 0.13 MB/sec execute 430 sec latency 307.216 ms 2024-01-20T08:42:11.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:42:11.914 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:42:11.987 INFO:tasks.workunit.client.0.smithi087.stdout: 1 13985 0.13 MB/sec execute 431 sec latency 304.757 ms 2024-01-20T08:42:12.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14001 0.13 MB/sec execute 432 sec latency 304.877 ms 2024-01-20T08:42:13.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14026 0.13 MB/sec execute 433 sec latency 135.280 ms 2024-01-20T08:42:14.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14044 0.13 MB/sec execute 434 sec latency 296.307 ms 2024-01-20T08:42:15.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14149 0.14 MB/sec execute 435 sec latency 308.605 ms 2024-01-20T08:42:16.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14179 0.14 MB/sec execute 436 sec latency 328.734 ms 2024-01-20T08:42:17.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14203 0.14 MB/sec execute 437 sec latency 239.527 ms 2024-01-20T08:42:18.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14218 0.14 MB/sec execute 438 sec latency 361.006 ms 2024-01-20T08:42:19.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14268 0.14 MB/sec execute 439 sec latency 333.331 ms 2024-01-20T08:42:20.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14285 0.14 MB/sec execute 440 sec latency 297.695 ms 2024-01-20T08:42:21.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14313 0.14 MB/sec execute 441 sec latency 237.680 ms 2024-01-20T08:42:22.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14326 0.14 MB/sec execute 442 sec latency 515.429 ms 2024-01-20T08:42:23.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14349 0.14 MB/sec execute 443 sec latency 313.803 ms 2024-01-20T08:42:24.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14399 0.14 MB/sec execute 444 sec latency 293.140 ms 2024-01-20T08:42:25.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14419 0.14 MB/sec execute 445 sec latency 251.132 ms 2024-01-20T08:42:26.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14430 0.14 MB/sec execute 446 sec latency 222.461 ms 2024-01-20T08:42:27.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14443 0.14 MB/sec execute 447 sec latency 262.395 ms 2024-01-20T08:42:28.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14459 0.14 MB/sec execute 448 sec latency 154.435 ms 2024-01-20T08:42:29.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14475 0.14 MB/sec execute 449 sec latency 182.780 ms 2024-01-20T08:42:30.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14491 0.14 MB/sec execute 450 sec latency 165.221 ms 2024-01-20T08:42:31.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14507 0.14 MB/sec execute 451 sec latency 165.803 ms 2024-01-20T08:42:32.988 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14517 0.14 MB/sec execute 452 sec latency 320.851 ms 2024-01-20T08:42:33.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14537 0.14 MB/sec execute 453 sec latency 286.033 ms 2024-01-20T08:42:34.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14554 0.14 MB/sec execute 454 sec latency 295.847 ms 2024-01-20T08:42:35.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14571 0.14 MB/sec execute 455 sec latency 155.305 ms 2024-01-20T08:42:36.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14582 0.14 MB/sec execute 456 sec latency 191.781 ms 2024-01-20T08:42:37.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14597 0.14 MB/sec execute 457 sec latency 231.031 ms 2024-01-20T08:42:38.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14614 0.14 MB/sec execute 458 sec latency 153.183 ms 2024-01-20T08:42:39.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14630 0.14 MB/sec execute 459 sec latency 162.973 ms 2024-01-20T08:42:40.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14647 0.13 MB/sec execute 460 sec latency 173.884 ms 2024-01-20T08:42:41.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14664 0.13 MB/sec execute 461 sec latency 151.788 ms 2024-01-20T08:42:42.051 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:42:42.053 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:42:42.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14671 0.13 MB/sec execute 462 sec latency 330.208 ms 2024-01-20T08:42:43.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14686 0.13 MB/sec execute 463 sec latency 308.741 ms 2024-01-20T08:42:44.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14702 0.13 MB/sec execute 464 sec latency 303.068 ms 2024-01-20T08:42:45.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14722 0.13 MB/sec execute 465 sec latency 301.270 ms 2024-01-20T08:42:46.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14748 0.13 MB/sec execute 466 sec latency 191.528 ms 2024-01-20T08:42:47.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14769 0.13 MB/sec execute 467 sec latency 225.980 ms 2024-01-20T08:42:48.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14789 0.13 MB/sec execute 468 sec latency 358.873 ms 2024-01-20T08:42:49.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14809 0.13 MB/sec execute 469 sec latency 425.892 ms 2024-01-20T08:42:50.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14817 0.13 MB/sec execute 470 sec latency 314.518 ms 2024-01-20T08:42:51.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14853 0.13 MB/sec execute 471 sec latency 342.654 ms 2024-01-20T08:42:52.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14862 0.13 MB/sec execute 472 sec latency 476.614 ms 2024-01-20T08:42:53.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14890 0.13 MB/sec execute 473 sec latency 293.782 ms 2024-01-20T08:42:54.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14925 0.13 MB/sec execute 474 sec latency 290.489 ms 2024-01-20T08:42:55.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14953 0.13 MB/sec execute 475 sec latency 334.103 ms 2024-01-20T08:42:56.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14996 0.13 MB/sec execute 476 sec latency 288.830 ms 2024-01-20T08:42:57.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15020 0.13 MB/sec execute 477 sec latency 324.890 ms 2024-01-20T08:42:58.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15059 0.13 MB/sec execute 478 sec latency 307.606 ms 2024-01-20T08:42:59.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15068 0.13 MB/sec execute 479 sec latency 293.638 ms 2024-01-20T08:43:00.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15094 0.13 MB/sec execute 480 sec latency 352.484 ms 2024-01-20T08:43:01.989 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15129 0.13 MB/sec execute 481 sec latency 228.167 ms 2024-01-20T08:43:02.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15136 0.13 MB/sec execute 482 sec latency 428.578 ms 2024-01-20T08:43:03.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15147 0.13 MB/sec execute 483 sec latency 291.186 ms 2024-01-20T08:43:04.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15176 0.13 MB/sec execute 484 sec latency 297.963 ms 2024-01-20T08:43:05.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15232 0.13 MB/sec execute 485 sec latency 303.984 ms 2024-01-20T08:43:06.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15250 0.13 MB/sec execute 486 sec latency 301.896 ms 2024-01-20T08:43:07.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15254 0.13 MB/sec execute 487 sec latency 313.056 ms 2024-01-20T08:43:08.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15292 0.13 MB/sec execute 488 sec latency 156.557 ms 2024-01-20T08:43:09.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15317 0.13 MB/sec execute 489 sec latency 293.993 ms 2024-01-20T08:43:10.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15350 0.13 MB/sec execute 490 sec latency 304.762 ms 2024-01-20T08:43:11.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15370 0.13 MB/sec execute 491 sec latency 291.948 ms 2024-01-20T08:43:12.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:43:12.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:43:12.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15375 0.13 MB/sec execute 492 sec latency 445.242 ms 2024-01-20T08:43:13.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15406 0.13 MB/sec execute 493 sec latency 258.991 ms 2024-01-20T08:43:14.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15433 0.13 MB/sec execute 494 sec latency 288.683 ms 2024-01-20T08:43:15.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15458 0.13 MB/sec execute 495 sec latency 308.637 ms 2024-01-20T08:43:16.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15483 0.13 MB/sec execute 496 sec latency 302.731 ms 2024-01-20T08:43:17.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15578 0.13 MB/sec execute 497 sec latency 418.041 ms 2024-01-20T08:43:18.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15587 0.13 MB/sec execute 498 sec latency 297.852 ms 2024-01-20T08:43:19.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15605 0.13 MB/sec execute 499 sec latency 314.410 ms 2024-01-20T08:43:20.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15630 0.13 MB/sec execute 500 sec latency 381.791 ms 2024-01-20T08:43:21.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15659 0.13 MB/sec execute 501 sec latency 281.151 ms 2024-01-20T08:43:22.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15687 0.13 MB/sec execute 502 sec latency 309.731 ms 2024-01-20T08:43:23.558 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:13:08.036 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.563 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:13:08.041 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.628 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:13:08.063 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.629 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:13:08.064 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.629 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:13:08.064 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.634 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:13:08.069 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.643 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:13:08.121 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.708 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:13:08.143 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.710 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:13:08.145 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:43:23.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15701 0.13 MB/sec execute 503 sec latency 356.661 ms 2024-01-20T08:43:24.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15728 0.13 MB/sec execute 504 sec latency 174.087 ms 2024-01-20T08:43:25.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15756 0.13 MB/sec execute 505 sec latency 304.347 ms 2024-01-20T08:43:26.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15783 0.13 MB/sec execute 506 sec latency 306.609 ms 2024-01-20T08:43:27.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15807 0.13 MB/sec execute 507 sec latency 317.290 ms 2024-01-20T08:43:28.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15818 0.13 MB/sec execute 508 sec latency 321.403 ms 2024-01-20T08:43:29.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15845 0.13 MB/sec execute 509 sec latency 319.923 ms 2024-01-20T08:43:30.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15887 0.13 MB/sec execute 510 sec latency 317.938 ms 2024-01-20T08:43:31.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15902 0.13 MB/sec execute 511 sec latency 260.993 ms 2024-01-20T08:43:32.990 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15963 0.13 MB/sec execute 512 sec latency 389.532 ms 2024-01-20T08:43:33.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16003 0.13 MB/sec execute 513 sec latency 296.022 ms 2024-01-20T08:43:34.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16023 0.13 MB/sec execute 514 sec latency 333.172 ms 2024-01-20T08:43:35.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16042 0.13 MB/sec execute 515 sec latency 308.689 ms 2024-01-20T08:43:36.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16063 0.13 MB/sec execute 516 sec latency 334.484 ms 2024-01-20T08:43:37.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16078 0.13 MB/sec execute 517 sec latency 324.966 ms 2024-01-20T08:43:38.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16110 0.13 MB/sec execute 518 sec latency 335.335 ms 2024-01-20T08:43:39.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16151 0.13 MB/sec execute 519 sec latency 281.369 ms 2024-01-20T08:43:40.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16181 0.13 MB/sec execute 520 sec latency 295.421 ms 2024-01-20T08:43:41.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16230 0.13 MB/sec execute 521 sec latency 311.109 ms 2024-01-20T08:43:42.252 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:43:42.255 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:43:42.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16249 0.13 MB/sec execute 522 sec latency 309.142 ms 2024-01-20T08:43:43.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16259 0.13 MB/sec execute 523 sec latency 300.597 ms 2024-01-20T08:43:44.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16279 0.13 MB/sec execute 524 sec latency 330.444 ms 2024-01-20T08:43:45.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16294 0.13 MB/sec execute 525 sec latency 324.033 ms 2024-01-20T08:43:46.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16344 0.13 MB/sec execute 526 sec latency 347.148 ms 2024-01-20T08:43:47.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16373 0.13 MB/sec execute 527 sec latency 351.015 ms 2024-01-20T08:43:48.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16382 0.13 MB/sec execute 528 sec latency 331.022 ms 2024-01-20T08:43:49.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16421 0.13 MB/sec execute 529 sec latency 303.151 ms 2024-01-20T08:43:50.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16491 0.13 MB/sec execute 530 sec latency 288.564 ms 2024-01-20T08:43:51.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16502 0.13 MB/sec execute 531 sec latency 436.937 ms 2024-01-20T08:43:52.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16520 0.13 MB/sec execute 532 sec latency 402.806 ms 2024-01-20T08:43:53.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16536 0.13 MB/sec execute 533 sec latency 287.668 ms 2024-01-20T08:43:54.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16558 0.13 MB/sec execute 534 sec latency 297.008 ms 2024-01-20T08:43:55.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16562 0.13 MB/sec execute 535 sec latency 560.174 ms 2024-01-20T08:43:56.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16584 0.13 MB/sec execute 536 sec latency 305.017 ms 2024-01-20T08:43:57.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16596 0.13 MB/sec execute 537 sec latency 312.576 ms 2024-01-20T08:43:58.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16624 0.13 MB/sec execute 538 sec latency 368.153 ms 2024-01-20T08:43:59.991 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16631 0.13 MB/sec execute 539 sec latency 323.142 ms 2024-01-20T08:44:00.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16650 0.13 MB/sec execute 540 sec latency 151.195 ms 2024-01-20T08:44:01.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16685 0.13 MB/sec execute 541 sec latency 352.386 ms 2024-01-20T08:44:02.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16696 0.13 MB/sec execute 542 sec latency 312.935 ms 2024-01-20T08:44:03.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16746 0.13 MB/sec execute 543 sec latency 318.682 ms 2024-01-20T08:44:04.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16761 0.13 MB/sec execute 544 sec latency 301.765 ms 2024-01-20T08:44:05.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16776 0.13 MB/sec execute 545 sec latency 332.444 ms 2024-01-20T08:44:06.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16795 0.13 MB/sec execute 546 sec latency 318.600 ms 2024-01-20T08:44:07.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16812 0.13 MB/sec execute 547 sec latency 342.633 ms 2024-01-20T08:44:08.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16817 0.13 MB/sec execute 548 sec latency 312.496 ms 2024-01-20T08:44:09.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16944 0.13 MB/sec execute 549 sec latency 314.390 ms 2024-01-20T08:44:10.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 16963 0.13 MB/sec execute 550 sec latency 334.453 ms 2024-01-20T08:44:11.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17064 0.13 MB/sec execute 551 sec latency 285.272 ms 2024-01-20T08:44:12.461 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:44:12.463 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:44:12.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17092 0.13 MB/sec execute 552 sec latency 322.827 ms 2024-01-20T08:44:13.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17124 0.13 MB/sec execute 553 sec latency 205.135 ms 2024-01-20T08:44:14.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17161 0.13 MB/sec execute 554 sec latency 304.785 ms 2024-01-20T08:44:15.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17176 0.13 MB/sec execute 555 sec latency 335.946 ms 2024-01-20T08:44:16.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17200 0.13 MB/sec execute 556 sec latency 312.933 ms 2024-01-20T08:44:17.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17223 0.13 MB/sec execute 557 sec latency 215.127 ms 2024-01-20T08:44:18.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17368 0.13 MB/sec execute 558 sec latency 304.237 ms 2024-01-20T08:44:19.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17398 0.13 MB/sec execute 559 sec latency 308.562 ms 2024-01-20T08:44:20.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17458 0.14 MB/sec execute 560 sec latency 281.023 ms 2024-01-20T08:44:21.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17473 0.14 MB/sec execute 561 sec latency 313.638 ms 2024-01-20T08:44:22.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17477 0.14 MB/sec execute 562 sec latency 365.205 ms 2024-01-20T08:44:23.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17482 0.14 MB/sec execute 563 sec latency 341.845 ms 2024-01-20T08:44:24.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17485 0.14 MB/sec execute 564 sec latency 398.344 ms 2024-01-20T08:44:25.992 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17489 0.14 MB/sec execute 565 sec latency 429.611 ms 2024-01-20T08:44:26.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17495 0.13 MB/sec execute 566 sec latency 313.190 ms 2024-01-20T08:44:27.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17500 0.13 MB/sec execute 567 sec latency 416.251 ms 2024-01-20T08:44:28.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17504 0.13 MB/sec execute 568 sec latency 319.400 ms 2024-01-20T08:44:29.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17508 0.13 MB/sec execute 569 sec latency 300.357 ms 2024-01-20T08:44:30.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17511 0.13 MB/sec execute 570 sec latency 300.073 ms 2024-01-20T08:44:31.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17516 0.13 MB/sec execute 571 sec latency 308.623 ms 2024-01-20T08:44:32.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17519 0.13 MB/sec execute 572 sec latency 329.726 ms 2024-01-20T08:44:33.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17524 0.13 MB/sec execute 573 sec latency 306.755 ms 2024-01-20T08:44:34.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17527 0.13 MB/sec execute 574 sec latency 302.468 ms 2024-01-20T08:44:35.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17544 0.13 MB/sec execute 575 sec latency 308.687 ms 2024-01-20T08:44:36.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17566 0.13 MB/sec execute 576 sec latency 134.580 ms 2024-01-20T08:44:37.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17585 0.13 MB/sec execute 577 sec latency 296.090 ms 2024-01-20T08:44:38.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17685 0.14 MB/sec execute 578 sec latency 317.125 ms 2024-01-20T08:44:39.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17720 0.14 MB/sec execute 579 sec latency 133.195 ms 2024-01-20T08:44:40.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17746 0.14 MB/sec execute 580 sec latency 281.226 ms 2024-01-20T08:44:41.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17761 0.14 MB/sec execute 581 sec latency 295.776 ms 2024-01-20T08:44:42.584 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:44:42.586 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:44:42.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17802 0.14 MB/sec execute 582 sec latency 421.316 ms 2024-01-20T08:44:43.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17826 0.14 MB/sec execute 583 sec latency 200.190 ms 2024-01-20T08:44:44.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17856 0.14 MB/sec execute 584 sec latency 322.521 ms 2024-01-20T08:44:45.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17869 0.14 MB/sec execute 585 sec latency 305.857 ms 2024-01-20T08:44:46.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17890 0.14 MB/sec execute 586 sec latency 305.021 ms 2024-01-20T08:44:47.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17941 0.14 MB/sec execute 587 sec latency 246.347 ms 2024-01-20T08:44:48.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17962 0.14 MB/sec execute 588 sec latency 302.925 ms 2024-01-20T08:44:49.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17973 0.14 MB/sec execute 589 sec latency 192.083 ms 2024-01-20T08:44:50.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17984 0.14 MB/sec execute 590 sec latency 206.436 ms 2024-01-20T08:44:51.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17999 0.14 MB/sec execute 591 sec latency 193.643 ms 2024-01-20T08:44:52.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18015 0.14 MB/sec execute 592 sec latency 145.221 ms 2024-01-20T08:44:53.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18030 0.14 MB/sec execute 593 sec latency 147.880 ms 2024-01-20T08:44:54.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18047 0.14 MB/sec execute 594 sec latency 164.927 ms 2024-01-20T08:44:55.993 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18059 0.14 MB/sec execute 595 sec latency 296.134 ms 2024-01-20T08:44:56.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18079 0.14 MB/sec execute 596 sec latency 342.801 ms 2024-01-20T08:44:57.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18097 0.14 MB/sec execute 597 sec latency 351.987 ms 2024-01-20T08:44:58.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18111 0.14 MB/sec execute 598 sec latency 310.494 ms 2024-01-20T08:44:59.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 18122 0.14 MB/sec execute 599 sec latency 170.068 ms 2024-01-20T08:45:00.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 600 sec 2024-01-20T08:45:01.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 601 sec 2024-01-20T08:45:02.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 602 sec 2024-01-20T08:45:03.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 603 sec 2024-01-20T08:45:04.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 604 sec 2024-01-20T08:45:05.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 605 sec 2024-01-20T08:45:06.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 606 sec 2024-01-20T08:45:07.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 607 sec 2024-01-20T08:45:08.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 608 sec 2024-01-20T08:45:09.994 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 609 sec 2024-01-20T08:45:10.350 INFO:tasks.workunit.client.0.smithi087.stdout: 0 cleanup 609 sec 2024-01-20T08:45:10.350 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-01-20T08:45:10.350 INFO:tasks.workunit.client.0.smithi087.stdout: Operation Count AvgLat MaxLat 2024-01-20T08:45:10.350 INFO:tasks.workunit.client.0.smithi087.stdout: ---------------------------------------- 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: NTCreateX 2583 64.104 262.387 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Close 1882 1.161 29.078 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Rename 108 179.315 346.917 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Unlink 530 71.817 403.815 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Qpathinfo 2323 35.031 258.983 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Qfileinfo 395 0.952 1.557 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Qfsinfo 434 10.384 112.993 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Sfileinfo 200 11.921 26.627 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Find 877 310.815 665.320 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: WriteX 1252 4.004 109.288 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: ReadX 3999 0.026 5.001 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: LockX 8 15.727 18.451 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: UnlockX 8 14.779 17.289 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: Flush 172 47.660 151.027 2024-01-20T08:45:10.351 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-01-20T08:45:10.352 INFO:tasks.workunit.client.0.smithi087.stdout:Throughput 0.135393 MB/sec 1 clients 1 procs max_latency=665.326 ms 2024-01-20T08:45:10.352 INFO:tasks.workunit.client.0.smithi087.stderr:+ dbench 10 2024-01-20T08:45:10.352 INFO:tasks.workunit.client.0.smithi087.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-20T08:45:10.352 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-01-20T08:45:10.352 INFO:tasks.workunit.client.0.smithi087.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-20T08:45:10.352 INFO:tasks.workunit.client.0.smithi087.stdout:8 of 10 processes prepared for launch 0 sec 2024-01-20T08:45:10.451 INFO:tasks.workunit.client.0.smithi087.stdout:10 of 10 processes prepared for launch 0 sec 2024-01-20T08:45:10.451 INFO:tasks.workunit.client.0.smithi087.stdout:releasing clients 2024-01-20T08:45:11.451 INFO:tasks.workunit.client.0.smithi087.stdout: 0 3 0.00 MB/sec warmup 1 sec latency 999.951 ms 2024-01-20T08:45:12.451 INFO:tasks.workunit.client.0.smithi087.stdout: 0 4 0.00 MB/sec warmup 2 sec latency 2000.010 ms 2024-01-20T08:45:12.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:45:12.709 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:45:13.451 INFO:tasks.workunit.client.0.smithi087.stdout: 4 12 1.16 MB/sec warmup 3 sec latency 3000.038 ms 2024-01-20T08:45:14.451 INFO:tasks.workunit.client.0.smithi087.stdout: 4 23 1.97 MB/sec warmup 4 sec latency 655.498 ms 2024-01-20T08:45:15.451 INFO:tasks.workunit.client.0.smithi087.stdout: 4 35 2.70 MB/sec warmup 5 sec latency 711.576 ms 2024-01-20T08:45:16.452 INFO:tasks.workunit.client.0.smithi087.stdout: 7 48 3.25 MB/sec warmup 6 sec latency 899.237 ms 2024-01-20T08:45:17.451 INFO:tasks.workunit.client.0.smithi087.stdout: 10 65 4.05 MB/sec warmup 7 sec latency 865.878 ms 2024-01-20T08:45:18.451 INFO:tasks.workunit.client.0.smithi087.stdout: 10 79 4.38 MB/sec warmup 8 sec latency 787.144 ms 2024-01-20T08:45:19.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 90 4.47 MB/sec warmup 9 sec latency 1094.760 ms 2024-01-20T08:45:20.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 105 4.78 MB/sec warmup 10 sec latency 1100.357 ms 2024-01-20T08:45:21.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 117 4.81 MB/sec warmup 11 sec latency 1199.215 ms 2024-01-20T08:45:22.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 125 4.69 MB/sec warmup 12 sec latency 1030.456 ms 2024-01-20T08:45:23.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 130 4.45 MB/sec warmup 13 sec latency 1221.486 ms 2024-01-20T08:45:24.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 134 4.23 MB/sec warmup 14 sec latency 1359.691 ms 2024-01-20T08:45:25.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 143 4.12 MB/sec warmup 15 sec latency 1342.767 ms 2024-01-20T08:45:26.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 147 3.94 MB/sec warmup 16 sec latency 1283.605 ms 2024-01-20T08:45:27.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 150 3.76 MB/sec warmup 17 sec latency 1421.981 ms 2024-01-20T08:45:28.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 157 3.65 MB/sec warmup 18 sec latency 1578.626 ms 2024-01-20T08:45:29.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 162 3.55 MB/sec warmup 19 sec latency 1200.051 ms 2024-01-20T08:45:30.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 168 3.47 MB/sec warmup 20 sec latency 1578.028 ms 2024-01-20T08:45:31.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 172 3.36 MB/sec warmup 21 sec latency 1469.225 ms 2024-01-20T08:45:32.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 176 3.26 MB/sec warmup 22 sec latency 1694.197 ms 2024-01-20T08:45:33.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 181 3.18 MB/sec warmup 23 sec latency 1530.779 ms 2024-01-20T08:45:34.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 189 3.15 MB/sec warmup 24 sec latency 1414.807 ms 2024-01-20T08:45:35.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 192 3.06 MB/sec warmup 25 sec latency 1270.389 ms 2024-01-20T08:45:36.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 198 3.02 MB/sec warmup 26 sec latency 1513.132 ms 2024-01-20T08:45:37.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 205 2.99 MB/sec warmup 27 sec latency 1349.203 ms 2024-01-20T08:45:38.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 209 2.93 MB/sec warmup 28 sec latency 1440.559 ms 2024-01-20T08:45:39.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 214 2.88 MB/sec warmup 29 sec latency 1550.928 ms 2024-01-20T08:45:40.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 221 2.86 MB/sec warmup 30 sec latency 1468.719 ms 2024-01-20T08:45:41.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 229 2.85 MB/sec warmup 31 sec latency 1379.463 ms 2024-01-20T08:45:42.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 231 2.78 MB/sec warmup 32 sec latency 1330.571 ms 2024-01-20T08:45:42.782 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:45:42.785 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:45:43.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 237 2.76 MB/sec warmup 33 sec latency 1631.241 ms 2024-01-20T08:45:44.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 244 2.74 MB/sec warmup 34 sec latency 1779.830 ms 2024-01-20T08:45:45.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 246 2.68 MB/sec warmup 35 sec latency 1583.054 ms 2024-01-20T08:45:46.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 254 2.68 MB/sec warmup 36 sec latency 1664.409 ms 2024-01-20T08:45:47.452 INFO:tasks.workunit.client.0.smithi087.stdout: 10 257 2.63 MB/sec warmup 37 sec latency 1768.234 ms 2024-01-20T08:45:48.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 259 2.58 MB/sec warmup 38 sec latency 2373.688 ms 2024-01-20T08:45:49.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 266 2.57 MB/sec warmup 39 sec latency 2585.108 ms 2024-01-20T08:45:50.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 270 2.54 MB/sec warmup 40 sec latency 2001.732 ms 2024-01-20T08:45:51.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 275 2.51 MB/sec warmup 41 sec latency 1679.461 ms 2024-01-20T08:45:52.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 280 2.49 MB/sec warmup 42 sec latency 1938.334 ms 2024-01-20T08:45:53.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 287 2.48 MB/sec warmup 43 sec latency 1811.334 ms 2024-01-20T08:45:54.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 290 2.45 MB/sec warmup 44 sec latency 1875.201 ms 2024-01-20T08:45:55.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 297 2.45 MB/sec warmup 45 sec latency 1625.897 ms 2024-01-20T08:45:56.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 300 2.41 MB/sec warmup 46 sec latency 1531.385 ms 2024-01-20T08:45:57.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 306 2.40 MB/sec warmup 47 sec latency 1818.606 ms 2024-01-20T08:45:58.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 310 2.37 MB/sec warmup 48 sec latency 1765.756 ms 2024-01-20T08:45:59.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 315 2.35 MB/sec warmup 49 sec latency 1593.235 ms 2024-01-20T08:46:00.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 322 2.35 MB/sec warmup 50 sec latency 1513.072 ms 2024-01-20T08:46:01.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 325 2.32 MB/sec warmup 51 sec latency 1619.213 ms 2024-01-20T08:46:02.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 330 2.30 MB/sec warmup 52 sec latency 1906.083 ms 2024-01-20T08:46:03.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 334 2.28 MB/sec warmup 53 sec latency 1679.859 ms 2024-01-20T08:46:04.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 341 2.28 MB/sec warmup 54 sec latency 1669.372 ms 2024-01-20T08:46:05.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 346 2.27 MB/sec warmup 55 sec latency 1442.300 ms 2024-01-20T08:46:06.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 352 2.26 MB/sec warmup 56 sec latency 1458.210 ms 2024-01-20T08:46:07.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 358 2.26 MB/sec warmup 57 sec latency 1599.387 ms 2024-01-20T08:46:08.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 361 2.23 MB/sec warmup 58 sec latency 1745.035 ms 2024-01-20T08:46:09.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 366 2.22 MB/sec warmup 59 sec latency 2129.203 ms 2024-01-20T08:46:10.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 371 2.21 MB/sec warmup 60 sec latency 2190.636 ms 2024-01-20T08:46:11.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 378 2.21 MB/sec warmup 61 sec latency 1638.192 ms 2024-01-20T08:46:12.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 383 2.20 MB/sec warmup 62 sec latency 1767.448 ms 2024-01-20T08:46:12.911 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:46:12.913 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:46:13.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 387 2.19 MB/sec warmup 63 sec latency 1949.556 ms 2024-01-20T08:46:14.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 393 2.18 MB/sec warmup 64 sec latency 1864.892 ms 2024-01-20T08:46:15.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 396 2.16 MB/sec warmup 65 sec latency 1772.849 ms 2024-01-20T08:46:16.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 402 2.16 MB/sec warmup 66 sec latency 1861.628 ms 2024-01-20T08:46:17.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 407 2.15 MB/sec warmup 67 sec latency 1793.136 ms 2024-01-20T08:46:18.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 412 2.14 MB/sec warmup 68 sec latency 2077.375 ms 2024-01-20T08:46:19.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 414 2.12 MB/sec warmup 69 sec latency 2231.950 ms 2024-01-20T08:46:20.453 INFO:tasks.workunit.client.0.smithi087.stdout: 10 418 2.11 MB/sec warmup 70 sec latency 2447.159 ms 2024-01-20T08:46:21.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 425 2.11 MB/sec warmup 71 sec latency 2639.368 ms 2024-01-20T08:46:22.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 428 2.10 MB/sec warmup 72 sec latency 1888.223 ms 2024-01-20T08:46:23.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 433 2.09 MB/sec warmup 73 sec latency 2061.226 ms 2024-01-20T08:46:24.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 436 2.08 MB/sec warmup 74 sec latency 2045.986 ms 2024-01-20T08:46:25.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 443 2.08 MB/sec warmup 75 sec latency 2202.540 ms 2024-01-20T08:46:26.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 446 2.07 MB/sec warmup 76 sec latency 1875.623 ms 2024-01-20T08:46:27.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 452 2.06 MB/sec warmup 77 sec latency 1955.318 ms 2024-01-20T08:46:28.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 455 2.05 MB/sec warmup 78 sec latency 1837.148 ms 2024-01-20T08:46:29.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 460 2.04 MB/sec warmup 79 sec latency 1868.405 ms 2024-01-20T08:46:30.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 463 2.03 MB/sec warmup 80 sec latency 2185.112 ms 2024-01-20T08:46:31.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 468 2.02 MB/sec warmup 81 sec latency 2089.281 ms 2024-01-20T08:46:32.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 475 2.03 MB/sec warmup 82 sec latency 2120.018 ms 2024-01-20T08:46:33.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 478 2.01 MB/sec warmup 83 sec latency 1763.580 ms 2024-01-20T08:46:34.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 481 2.00 MB/sec warmup 84 sec latency 2303.365 ms 2024-01-20T08:46:35.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 487 2.01 MB/sec warmup 85 sec latency 2119.137 ms 2024-01-20T08:46:36.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 499 2.02 MB/sec warmup 86 sec latency 1714.767 ms 2024-01-20T08:46:37.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 501 2.01 MB/sec warmup 87 sec latency 1809.590 ms 2024-01-20T08:46:38.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 505 2.00 MB/sec warmup 88 sec latency 2604.244 ms 2024-01-20T08:46:39.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 507 1.99 MB/sec warmup 89 sec latency 3604.276 ms 2024-01-20T08:46:40.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 513 1.99 MB/sec warmup 90 sec latency 4604.309 ms 2024-01-20T08:46:41.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 523 2.00 MB/sec warmup 91 sec latency 4748.476 ms 2024-01-20T08:46:42.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 531 2.01 MB/sec warmup 92 sec latency 1818.227 ms 2024-01-20T08:46:43.035 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:46:43.038 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:46:43.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 535 2.01 MB/sec warmup 93 sec latency 1831.291 ms 2024-01-20T08:46:44.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 548 2.01 MB/sec warmup 94 sec latency 2560.377 ms 2024-01-20T08:46:45.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 561 2.05 MB/sec warmup 95 sec latency 3192.739 ms 2024-01-20T08:46:46.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 568 2.05 MB/sec warmup 96 sec latency 1587.708 ms 2024-01-20T08:46:47.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 574 2.05 MB/sec warmup 97 sec latency 2154.607 ms 2024-01-20T08:46:48.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 585 2.07 MB/sec warmup 98 sec latency 3154.633 ms 2024-01-20T08:46:49.454 INFO:tasks.workunit.client.0.smithi087.stdout: 10 592 2.07 MB/sec warmup 99 sec latency 4114.493 ms 2024-01-20T08:46:50.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 604 2.08 MB/sec warmup 100 sec latency 3115.631 ms 2024-01-20T08:46:51.070 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:16:35.548 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.072 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:16:35.550 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.124 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:16:35.559 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.125 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:16:35.561 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.127 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:16:35.563 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.127 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:16:35.563 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.158 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:16:35.636 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.205 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:16:35.641 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.206 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:16:35.642 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:46:51.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 611 2.08 MB/sec warmup 101 sec latency 3182.369 ms 2024-01-20T08:46:52.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 616 2.06 MB/sec warmup 102 sec latency 3302.989 ms 2024-01-20T08:46:53.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 617 2.04 MB/sec warmup 103 sec latency 3065.293 ms 2024-01-20T08:46:54.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 620 2.02 MB/sec warmup 104 sec latency 2886.543 ms 2024-01-20T08:46:55.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 628 2.00 MB/sec warmup 105 sec latency 2855.606 ms 2024-01-20T08:46:56.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 632 1.98 MB/sec warmup 106 sec latency 2818.719 ms 2024-01-20T08:46:57.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 633 1.97 MB/sec warmup 107 sec latency 2638.787 ms 2024-01-20T08:46:58.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 642 1.95 MB/sec warmup 108 sec latency 2713.448 ms 2024-01-20T08:46:59.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 643 1.93 MB/sec warmup 109 sec latency 3065.808 ms 2024-01-20T08:47:00.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 647 1.92 MB/sec warmup 110 sec latency 3052.263 ms 2024-01-20T08:47:01.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 650 1.90 MB/sec warmup 111 sec latency 3157.636 ms 2024-01-20T08:47:02.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 652 1.88 MB/sec warmup 112 sec latency 2737.770 ms 2024-01-20T08:47:03.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 653 1.87 MB/sec warmup 113 sec latency 2672.259 ms 2024-01-20T08:47:04.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 656 1.85 MB/sec warmup 114 sec latency 3084.822 ms 2024-01-20T08:47:05.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 659 1.83 MB/sec warmup 115 sec latency 2782.304 ms 2024-01-20T08:47:06.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 663 1.82 MB/sec warmup 116 sec latency 3234.230 ms 2024-01-20T08:47:07.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 667 1.81 MB/sec warmup 117 sec latency 3021.200 ms 2024-01-20T08:47:08.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 670 1.79 MB/sec warmup 118 sec latency 3162.141 ms 2024-01-20T08:47:09.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 673 1.78 MB/sec warmup 119 sec latency 3226.359 ms 2024-01-20T08:47:11.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 685 0.25 MB/sec execute 1 sec latency 3229.542 ms 2024-01-20T08:47:12.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 688 0.14 MB/sec execute 2 sec latency 3019.087 ms 2024-01-20T08:47:13.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:47:13.118 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:47:13.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 691 0.10 MB/sec execute 3 sec latency 2823.997 ms 2024-01-20T08:47:14.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 696 0.11 MB/sec execute 4 sec latency 3244.075 ms 2024-01-20T08:47:15.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 701 0.09 MB/sec execute 5 sec latency 3422.732 ms 2024-01-20T08:47:16.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 704 0.08 MB/sec execute 6 sec latency 3353.047 ms 2024-01-20T08:47:17.455 INFO:tasks.workunit.client.0.smithi087.stdout: 10 706 0.07 MB/sec execute 7 sec latency 3644.955 ms 2024-01-20T08:47:18.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 713 0.10 MB/sec execute 8 sec latency 3527.661 ms 2024-01-20T08:47:19.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 719 0.11 MB/sec execute 9 sec latency 3604.147 ms 2024-01-20T08:47:20.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 726 0.10 MB/sec execute 10 sec latency 3734.082 ms 2024-01-20T08:47:21.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 729 0.09 MB/sec execute 11 sec latency 4096.029 ms 2024-01-20T08:47:22.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 732 0.09 MB/sec execute 12 sec latency 1999.528 ms 2024-01-20T08:47:23.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 735 0.09 MB/sec execute 13 sec latency 2999.585 ms 2024-01-20T08:47:24.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 739 0.10 MB/sec execute 14 sec latency 3999.636 ms 2024-01-20T08:47:25.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 744 0.10 MB/sec execute 15 sec latency 4286.854 ms 2024-01-20T08:47:26.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 751 0.10 MB/sec execute 16 sec latency 2960.190 ms 2024-01-20T08:47:27.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 753 0.10 MB/sec execute 17 sec latency 3409.018 ms 2024-01-20T08:47:28.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 757 0.10 MB/sec execute 18 sec latency 3343.846 ms 2024-01-20T08:47:29.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 764 0.10 MB/sec execute 19 sec latency 3437.581 ms 2024-01-20T08:47:30.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 767 0.10 MB/sec execute 20 sec latency 3526.328 ms 2024-01-20T08:47:31.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 771 0.10 MB/sec execute 21 sec latency 3758.462 ms 2024-01-20T08:47:32.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 774 0.10 MB/sec execute 22 sec latency 3298.047 ms 2024-01-20T08:47:33.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 777 0.10 MB/sec execute 23 sec latency 3326.772 ms 2024-01-20T08:47:34.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 783 0.10 MB/sec execute 24 sec latency 3227.706 ms 2024-01-20T08:47:35.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 793 0.11 MB/sec execute 25 sec latency 2940.639 ms 2024-01-20T08:47:36.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 797 0.10 MB/sec execute 26 sec latency 1986.720 ms 2024-01-20T08:47:36.979 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:17:21.457 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.035 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:17:21.513 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.065 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:17:21.543 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.107 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:17:21.543 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.110 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:17:21.545 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.111 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:17:21.546 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.111 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:17:21.546 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.134 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:17:21.570 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.138 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:17:21.573 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:47:37.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 799 0.10 MB/sec execute 27 sec latency 2986.748 ms 2024-01-20T08:47:38.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 801 0.10 MB/sec execute 28 sec latency 3646.740 ms 2024-01-20T08:47:39.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 804 0.10 MB/sec execute 29 sec latency 3629.774 ms 2024-01-20T08:47:40.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 809 0.10 MB/sec execute 30 sec latency 3510.252 ms 2024-01-20T08:47:41.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 813 0.10 MB/sec execute 31 sec latency 3184.052 ms 2024-01-20T08:47:42.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 815 0.10 MB/sec execute 32 sec latency 3054.183 ms 2024-01-20T08:47:43.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:47:43.257 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:47:43.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 819 0.10 MB/sec execute 33 sec latency 2868.384 ms 2024-01-20T08:47:44.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 819 0.09 MB/sec execute 34 sec latency 2920.635 ms 2024-01-20T08:47:45.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 824 0.10 MB/sec execute 35 sec latency 2809.066 ms 2024-01-20T08:47:46.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 826 0.09 MB/sec execute 36 sec latency 2467.811 ms 2024-01-20T08:47:47.456 INFO:tasks.workunit.client.0.smithi087.stdout: 10 830 0.09 MB/sec execute 37 sec latency 2276.022 ms 2024-01-20T08:47:48.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 836 0.09 MB/sec execute 38 sec latency 2601.928 ms 2024-01-20T08:47:49.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 841 0.10 MB/sec execute 39 sec latency 2490.380 ms 2024-01-20T08:47:50.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 842 0.09 MB/sec execute 40 sec latency 2551.122 ms 2024-01-20T08:47:51.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 845 0.09 MB/sec execute 41 sec latency 2566.632 ms 2024-01-20T08:47:52.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 850 0.09 MB/sec execute 42 sec latency 2666.080 ms 2024-01-20T08:47:53.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 857 0.10 MB/sec execute 43 sec latency 2761.388 ms 2024-01-20T08:47:54.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 858 0.09 MB/sec execute 44 sec latency 3033.812 ms 2024-01-20T08:47:55.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 862 0.09 MB/sec execute 45 sec latency 3098.895 ms 2024-01-20T08:47:56.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 863 0.09 MB/sec execute 46 sec latency 2980.926 ms 2024-01-20T08:47:57.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 864 0.09 MB/sec execute 47 sec latency 2792.634 ms 2024-01-20T08:47:58.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 869 0.09 MB/sec execute 48 sec latency 2865.366 ms 2024-01-20T08:47:59.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 872 0.09 MB/sec execute 49 sec latency 3264.767 ms 2024-01-20T08:48:00.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 878 0.09 MB/sec execute 50 sec latency 3421.959 ms 2024-01-20T08:48:01.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 879 0.09 MB/sec execute 51 sec latency 3031.770 ms 2024-01-20T08:48:02.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 882 0.09 MB/sec execute 52 sec latency 3419.911 ms 2024-01-20T08:48:03.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 888 0.09 MB/sec execute 53 sec latency 3039.047 ms 2024-01-20T08:48:04.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 892 0.09 MB/sec execute 54 sec latency 2854.650 ms 2024-01-20T08:48:05.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 894 0.09 MB/sec execute 55 sec latency 2081.570 ms 2024-01-20T08:48:06.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 898 0.10 MB/sec execute 56 sec latency 3081.596 ms 2024-01-20T08:48:07.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 902 0.10 MB/sec execute 57 sec latency 3959.404 ms 2024-01-20T08:48:08.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 908 0.10 MB/sec execute 58 sec latency 3850.925 ms 2024-01-20T08:48:09.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 910 0.10 MB/sec execute 59 sec latency 2509.844 ms 2024-01-20T08:48:10.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 915 0.09 MB/sec execute 60 sec latency 3226.272 ms 2024-01-20T08:48:11.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 916 0.09 MB/sec execute 61 sec latency 3570.846 ms 2024-01-20T08:48:12.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 920 0.09 MB/sec execute 62 sec latency 3575.792 ms 2024-01-20T08:48:13.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:48:13.398 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:48:13.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 922 0.09 MB/sec execute 63 sec latency 2890.125 ms 2024-01-20T08:48:14.457 INFO:tasks.workunit.client.0.smithi087.stdout: 10 924 0.09 MB/sec execute 64 sec latency 3699.609 ms 2024-01-20T08:48:15.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 927 0.09 MB/sec execute 65 sec latency 3637.007 ms 2024-01-20T08:48:16.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 929 0.09 MB/sec execute 66 sec latency 3622.669 ms 2024-01-20T08:48:17.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 929 0.09 MB/sec execute 67 sec latency 3205.061 ms 2024-01-20T08:48:18.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 931 0.09 MB/sec execute 68 sec latency 2645.040 ms 2024-01-20T08:48:19.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 931 0.09 MB/sec execute 69 sec latency 2455.395 ms 2024-01-20T08:48:20.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 934 0.09 MB/sec execute 70 sec latency 2652.061 ms 2024-01-20T08:48:21.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 936 0.09 MB/sec execute 71 sec latency 2438.410 ms 2024-01-20T08:48:22.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 937 0.08 MB/sec execute 72 sec latency 2729.723 ms 2024-01-20T08:48:23.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 941 0.09 MB/sec execute 73 sec latency 2300.248 ms 2024-01-20T08:48:24.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 944 0.08 MB/sec execute 74 sec latency 2853.219 ms 2024-01-20T08:48:25.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 945 0.08 MB/sec execute 75 sec latency 2923.685 ms 2024-01-20T08:48:26.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 948 0.08 MB/sec execute 76 sec latency 3026.918 ms 2024-01-20T08:48:27.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 954 0.09 MB/sec execute 77 sec latency 3076.405 ms 2024-01-20T08:48:28.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 961 0.10 MB/sec execute 78 sec latency 2594.450 ms 2024-01-20T08:48:29.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 962 0.09 MB/sec execute 79 sec latency 2918.970 ms 2024-01-20T08:48:30.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 966 0.09 MB/sec execute 80 sec latency 3114.209 ms 2024-01-20T08:48:31.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 974 0.10 MB/sec execute 81 sec latency 3255.512 ms 2024-01-20T08:48:32.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 982 0.11 MB/sec execute 82 sec latency 2346.972 ms 2024-01-20T08:48:33.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 985 0.11 MB/sec execute 83 sec latency 2071.687 ms 2024-01-20T08:48:34.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 987 0.11 MB/sec execute 84 sec latency 3071.714 ms 2024-01-20T08:48:35.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 990 0.11 MB/sec execute 85 sec latency 3671.596 ms 2024-01-20T08:48:36.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 996 0.11 MB/sec execute 86 sec latency 4055.360 ms 2024-01-20T08:48:37.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1005 0.12 MB/sec execute 87 sec latency 1546.223 ms 2024-01-20T08:48:38.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1011 0.14 MB/sec execute 88 sec latency 2546.274 ms 2024-01-20T08:48:39.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1012 0.14 MB/sec execute 89 sec latency 3546.307 ms 2024-01-20T08:48:40.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1017 0.15 MB/sec execute 90 sec latency 3578.663 ms 2024-01-20T08:48:41.458 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1024 0.17 MB/sec execute 91 sec latency 3205.235 ms 2024-01-20T08:48:42.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1027 0.17 MB/sec execute 92 sec latency 2571.902 ms 2024-01-20T08:48:43.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1032 0.18 MB/sec execute 93 sec latency 2590.262 ms 2024-01-20T08:48:43.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:48:43.537 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:48:44.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1033 0.18 MB/sec execute 94 sec latency 2740.177 ms 2024-01-20T08:48:45.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1037 0.18 MB/sec execute 95 sec latency 3007.599 ms 2024-01-20T08:48:46.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1039 0.18 MB/sec execute 96 sec latency 2792.885 ms 2024-01-20T08:48:47.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1041 0.18 MB/sec execute 97 sec latency 2732.667 ms 2024-01-20T08:48:48.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1043 0.18 MB/sec execute 98 sec latency 2971.069 ms 2024-01-20T08:48:49.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1045 0.17 MB/sec execute 99 sec latency 2883.754 ms 2024-01-20T08:48:50.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1049 0.17 MB/sec execute 100 sec latency 2832.559 ms 2024-01-20T08:48:51.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1051 0.17 MB/sec execute 101 sec latency 2762.075 ms 2024-01-20T08:48:52.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1052 0.17 MB/sec execute 102 sec latency 2891.738 ms 2024-01-20T08:48:53.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1053 0.17 MB/sec execute 103 sec latency 2666.908 ms 2024-01-20T08:48:54.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1056 0.17 MB/sec execute 104 sec latency 3197.263 ms 2024-01-20T08:48:55.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1057 0.17 MB/sec execute 105 sec latency 3227.612 ms 2024-01-20T08:48:56.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1060 0.16 MB/sec execute 106 sec latency 3347.158 ms 2024-01-20T08:48:57.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1062 0.16 MB/sec execute 107 sec latency 3510.796 ms 2024-01-20T08:48:58.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1067 0.16 MB/sec execute 108 sec latency 2986.754 ms 2024-01-20T08:48:59.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1073 0.16 MB/sec execute 109 sec latency 2143.302 ms 2024-01-20T08:49:00.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1075 0.16 MB/sec execute 110 sec latency 3143.353 ms 2024-01-20T08:49:01.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1078 0.16 MB/sec execute 111 sec latency 3572.029 ms 2024-01-20T08:49:02.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1081 0.16 MB/sec execute 112 sec latency 3498.624 ms 2024-01-20T08:49:03.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1087 0.16 MB/sec execute 113 sec latency 3505.258 ms 2024-01-20T08:49:04.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1091 0.16 MB/sec execute 114 sec latency 4028.084 ms 2024-01-20T08:49:05.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1093 0.15 MB/sec execute 115 sec latency 2844.206 ms 2024-01-20T08:49:06.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1097 0.15 MB/sec execute 116 sec latency 3844.234 ms 2024-01-20T08:49:07.459 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1101 0.15 MB/sec execute 117 sec latency 4052.636 ms 2024-01-20T08:49:08.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1104 0.15 MB/sec execute 118 sec latency 3672.129 ms 2024-01-20T08:49:09.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1108 0.15 MB/sec execute 119 sec latency 3487.647 ms 2024-01-20T08:49:10.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1110 0.15 MB/sec execute 120 sec latency 2906.476 ms 2024-01-20T08:49:11.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1113 0.15 MB/sec execute 121 sec latency 3398.991 ms 2024-01-20T08:49:12.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1118 0.15 MB/sec execute 122 sec latency 3230.193 ms 2024-01-20T08:49:13.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1122 0.15 MB/sec execute 123 sec latency 2962.355 ms 2024-01-20T08:49:13.679 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:49:13.682 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:49:14.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1125 0.15 MB/sec execute 124 sec latency 2459.052 ms 2024-01-20T08:49:15.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1130 0.14 MB/sec execute 125 sec latency 3009.537 ms 2024-01-20T08:49:16.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1136 0.14 MB/sec execute 126 sec latency 3087.972 ms 2024-01-20T08:49:17.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1139 0.14 MB/sec execute 127 sec latency 1794.895 ms 2024-01-20T08:49:18.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1144 0.14 MB/sec execute 128 sec latency 2794.922 ms 2024-01-20T08:49:19.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1146 0.14 MB/sec execute 129 sec latency 2838.755 ms 2024-01-20T08:49:20.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1151 0.14 MB/sec execute 130 sec latency 2702.323 ms 2024-01-20T08:49:21.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1156 0.14 MB/sec execute 131 sec latency 2855.910 ms 2024-01-20T08:49:22.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1160 0.14 MB/sec execute 132 sec latency 2444.774 ms 2024-01-20T08:49:23.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1162 0.14 MB/sec execute 133 sec latency 3444.829 ms 2024-01-20T08:49:24.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1164 0.14 MB/sec execute 134 sec latency 3567.158 ms 2024-01-20T08:49:25.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1165 0.14 MB/sec execute 135 sec latency 3562.458 ms 2024-01-20T08:49:26.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1169 0.14 MB/sec execute 136 sec latency 3334.834 ms 2024-01-20T08:49:27.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1170 0.13 MB/sec execute 137 sec latency 3110.401 ms 2024-01-20T08:49:28.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1172 0.13 MB/sec execute 138 sec latency 2954.797 ms 2024-01-20T08:49:29.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1173 0.13 MB/sec execute 139 sec latency 3196.956 ms 2024-01-20T08:49:30.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1174 0.13 MB/sec execute 140 sec latency 3315.502 ms 2024-01-20T08:49:31.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1178 0.13 MB/sec execute 141 sec latency 3189.460 ms 2024-01-20T08:49:32.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1180 0.13 MB/sec execute 142 sec latency 3112.463 ms 2024-01-20T08:49:33.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1183 0.13 MB/sec execute 143 sec latency 2839.206 ms 2024-01-20T08:49:34.460 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1184 0.13 MB/sec execute 144 sec latency 2930.436 ms 2024-01-20T08:49:35.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1187 0.13 MB/sec execute 145 sec latency 3520.848 ms 2024-01-20T08:49:36.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1193 0.13 MB/sec execute 146 sec latency 3444.052 ms 2024-01-20T08:49:37.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1196 0.13 MB/sec execute 147 sec latency 3255.201 ms 2024-01-20T08:49:38.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1199 0.13 MB/sec execute 148 sec latency 3001.848 ms 2024-01-20T08:49:39.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1203 0.13 MB/sec execute 149 sec latency 3095.481 ms 2024-01-20T08:49:40.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1206 0.13 MB/sec execute 150 sec latency 3345.433 ms 2024-01-20T08:49:41.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1209 0.12 MB/sec execute 151 sec latency 3939.507 ms 2024-01-20T08:49:42.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1215 0.12 MB/sec execute 152 sec latency 3440.276 ms 2024-01-20T08:49:43.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1221 0.12 MB/sec execute 153 sec latency 2583.393 ms 2024-01-20T08:49:43.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:49:43.737 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:49:44.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1226 0.12 MB/sec execute 154 sec latency 1922.937 ms 2024-01-20T08:49:45.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1228 0.12 MB/sec execute 155 sec latency 2922.968 ms 2024-01-20T08:49:46.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1234 0.12 MB/sec execute 156 sec latency 3746.649 ms 2024-01-20T08:49:47.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1236 0.12 MB/sec execute 157 sec latency 3170.336 ms 2024-01-20T08:49:48.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1238 0.12 MB/sec execute 158 sec latency 3661.013 ms 2024-01-20T08:49:49.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1240 0.12 MB/sec execute 159 sec latency 3265.957 ms 2024-01-20T08:49:50.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1245 0.12 MB/sec execute 160 sec latency 3666.232 ms 2024-01-20T08:49:51.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1249 0.12 MB/sec execute 161 sec latency 2974.971 ms 2024-01-20T08:49:52.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1256 0.12 MB/sec execute 162 sec latency 3220.902 ms 2024-01-20T08:49:53.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1260 0.12 MB/sec execute 163 sec latency 3523.440 ms 2024-01-20T08:49:54.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1266 0.12 MB/sec execute 164 sec latency 3985.636 ms 2024-01-20T08:49:55.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1275 0.14 MB/sec execute 165 sec latency 4088.607 ms 2024-01-20T08:49:56.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1276 0.13 MB/sec execute 166 sec latency 4497.494 ms 2024-01-20T08:49:57.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1277 0.13 MB/sec execute 167 sec latency 3830.833 ms 2024-01-20T08:49:58.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1278 0.13 MB/sec execute 168 sec latency 2850.567 ms 2024-01-20T08:49:59.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1281 0.13 MB/sec execute 169 sec latency 3083.292 ms 2024-01-20T08:50:00.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1288 0.13 MB/sec execute 170 sec latency 3489.111 ms 2024-01-20T08:50:01.461 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1293 0.14 MB/sec execute 171 sec latency 3743.662 ms 2024-01-20T08:50:02.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1299 0.15 MB/sec execute 172 sec latency 3911.096 ms 2024-01-20T08:50:03.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1300 0.15 MB/sec execute 173 sec latency 3362.185 ms 2024-01-20T08:50:04.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1301 0.15 MB/sec execute 174 sec latency 2595.372 ms 2024-01-20T08:50:05.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1311 0.16 MB/sec execute 175 sec latency 2646.612 ms 2024-01-20T08:50:06.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1321 0.18 MB/sec execute 176 sec latency 3106.951 ms 2024-01-20T08:50:07.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1339 0.21 MB/sec execute 177 sec latency 3080.706 ms 2024-01-20T08:50:08.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1340 0.21 MB/sec execute 178 sec latency 2737.670 ms 2024-01-20T08:50:09.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1342 0.21 MB/sec execute 179 sec latency 2582.622 ms 2024-01-20T08:50:10.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1342 0.20 MB/sec execute 180 sec latency 2425.757 ms 2024-01-20T08:50:11.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1346 0.21 MB/sec execute 181 sec latency 2512.836 ms 2024-01-20T08:50:12.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1381 0.27 MB/sec execute 182 sec latency 2607.661 ms 2024-01-20T08:50:13.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1383 0.26 MB/sec execute 183 sec latency 2511.311 ms 2024-01-20T08:50:13.798 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:50:13.801 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:50:14.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1384 0.26 MB/sec execute 184 sec latency 2587.126 ms 2024-01-20T08:50:15.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1385 0.26 MB/sec execute 185 sec latency 2895.064 ms 2024-01-20T08:50:16.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1386 0.26 MB/sec execute 186 sec latency 2611.581 ms 2024-01-20T08:50:17.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1389 0.26 MB/sec execute 187 sec latency 2437.328 ms 2024-01-20T08:50:18.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1394 0.26 MB/sec execute 188 sec latency 2628.029 ms 2024-01-20T08:50:19.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1396 0.26 MB/sec execute 189 sec latency 3107.105 ms 2024-01-20T08:50:20.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1397 0.25 MB/sec execute 190 sec latency 3052.053 ms 2024-01-20T08:50:21.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1398 0.25 MB/sec execute 191 sec latency 3091.046 ms 2024-01-20T08:50:22.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1403 0.25 MB/sec execute 192 sec latency 2697.038 ms 2024-01-20T08:50:23.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1407 0.25 MB/sec execute 193 sec latency 2535.591 ms 2024-01-20T08:50:24.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1413 0.25 MB/sec execute 194 sec latency 2721.912 ms 2024-01-20T08:50:25.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1415 0.25 MB/sec execute 195 sec latency 3186.718 ms 2024-01-20T08:50:26.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1416 0.25 MB/sec execute 196 sec latency 3236.140 ms 2024-01-20T08:50:27.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1420 0.25 MB/sec execute 197 sec latency 2965.276 ms 2024-01-20T08:50:28.462 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1426 0.24 MB/sec execute 198 sec latency 2394.379 ms 2024-01-20T08:50:29.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1428 0.24 MB/sec execute 199 sec latency 2924.112 ms 2024-01-20T08:50:30.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1435 0.24 MB/sec execute 200 sec latency 3093.040 ms 2024-01-20T08:50:31.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1438 0.24 MB/sec execute 201 sec latency 2412.650 ms 2024-01-20T08:50:32.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1441 0.24 MB/sec execute 202 sec latency 3200.942 ms 2024-01-20T08:50:33.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1443 0.24 MB/sec execute 203 sec latency 3021.480 ms 2024-01-20T08:50:34.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1446 0.24 MB/sec execute 204 sec latency 2972.230 ms 2024-01-20T08:50:35.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1448 0.24 MB/sec execute 205 sec latency 3331.397 ms 2024-01-20T08:50:36.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1450 0.24 MB/sec execute 206 sec latency 3625.798 ms 2024-01-20T08:50:37.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1455 0.23 MB/sec execute 207 sec latency 2166.236 ms 2024-01-20T08:50:38.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1459 0.23 MB/sec execute 208 sec latency 2654.814 ms 2024-01-20T08:50:39.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1463 0.23 MB/sec execute 209 sec latency 2849.766 ms 2024-01-20T08:50:40.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1467 0.23 MB/sec execute 210 sec latency 2959.980 ms 2024-01-20T08:50:41.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1470 0.23 MB/sec execute 211 sec latency 1181.119 ms 2024-01-20T08:50:42.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1470 0.23 MB/sec execute 212 sec latency 2181.148 ms 2024-01-20T08:50:43.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1474 0.23 MB/sec execute 213 sec latency 2807.548 ms 2024-01-20T08:50:44.128 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:50:44.130 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:50:44.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1481 0.23 MB/sec execute 214 sec latency 2464.258 ms 2024-01-20T08:50:45.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1482 0.23 MB/sec execute 215 sec latency 2930.549 ms 2024-01-20T08:50:46.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1486 0.23 MB/sec execute 216 sec latency 2971.666 ms 2024-01-20T08:50:47.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1488 0.23 MB/sec execute 217 sec latency 3263.702 ms 2024-01-20T08:50:48.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1491 0.22 MB/sec execute 218 sec latency 3507.073 ms 2024-01-20T08:50:49.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1495 0.22 MB/sec execute 219 sec latency 2726.146 ms 2024-01-20T08:50:50.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1499 0.22 MB/sec execute 220 sec latency 2926.171 ms 2024-01-20T08:50:51.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1500 0.22 MB/sec execute 221 sec latency 2698.010 ms 2024-01-20T08:50:52.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1504 0.22 MB/sec execute 222 sec latency 2681.113 ms 2024-01-20T08:50:53.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1506 0.22 MB/sec execute 223 sec latency 2823.082 ms 2024-01-20T08:50:54.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1510 0.22 MB/sec execute 224 sec latency 2846.965 ms 2024-01-20T08:50:55.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1514 0.22 MB/sec execute 225 sec latency 3581.479 ms 2024-01-20T08:50:56.463 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1519 0.22 MB/sec execute 226 sec latency 2438.840 ms 2024-01-20T08:50:57.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1522 0.22 MB/sec execute 227 sec latency 3438.895 ms 2024-01-20T08:50:58.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1526 0.22 MB/sec execute 228 sec latency 3528.364 ms 2024-01-20T08:50:59.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1527 0.21 MB/sec execute 229 sec latency 2929.789 ms 2024-01-20T08:51:00.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1529 0.21 MB/sec execute 230 sec latency 2633.925 ms 2024-01-20T08:51:01.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1535 0.21 MB/sec execute 231 sec latency 2688.148 ms 2024-01-20T08:51:02.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1540 0.21 MB/sec execute 232 sec latency 2640.954 ms 2024-01-20T08:51:03.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1542 0.21 MB/sec execute 233 sec latency 3156.968 ms 2024-01-20T08:51:04.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1544 0.21 MB/sec execute 234 sec latency 3563.953 ms 2024-01-20T08:51:05.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1548 0.21 MB/sec execute 235 sec latency 3894.594 ms 2024-01-20T08:51:06.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1552 0.21 MB/sec execute 236 sec latency 2811.602 ms 2024-01-20T08:51:07.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1555 0.21 MB/sec execute 237 sec latency 3156.439 ms 2024-01-20T08:51:08.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1557 0.21 MB/sec execute 238 sec latency 3233.690 ms 2024-01-20T08:51:09.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1560 0.21 MB/sec execute 239 sec latency 3477.118 ms 2024-01-20T08:51:10.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1562 0.21 MB/sec execute 240 sec latency 3660.231 ms 2024-01-20T08:51:11.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1566 0.21 MB/sec execute 241 sec latency 3521.325 ms 2024-01-20T08:51:12.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1571 0.20 MB/sec execute 242 sec latency 3572.275 ms 2024-01-20T08:51:13.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1576 0.20 MB/sec execute 243 sec latency 3497.991 ms 2024-01-20T08:51:14.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:51:14.204 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:51:14.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1580 0.20 MB/sec execute 244 sec latency 3826.198 ms 2024-01-20T08:51:15.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1583 0.20 MB/sec execute 245 sec latency 3423.792 ms 2024-01-20T08:51:16.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1587 0.20 MB/sec execute 246 sec latency 2790.118 ms 2024-01-20T08:51:17.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1590 0.20 MB/sec execute 247 sec latency 2974.151 ms 2024-01-20T08:51:18.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1594 0.20 MB/sec execute 248 sec latency 2463.005 ms 2024-01-20T08:51:19.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1597 0.20 MB/sec execute 249 sec latency 3117.510 ms 2024-01-20T08:51:20.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1601 0.20 MB/sec execute 250 sec latency 2978.007 ms 2024-01-20T08:51:21.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1604 0.20 MB/sec execute 251 sec latency 3205.447 ms 2024-01-20T08:51:22.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1606 0.20 MB/sec execute 252 sec latency 2809.417 ms 2024-01-20T08:51:23.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1610 0.20 MB/sec execute 253 sec latency 2504.546 ms 2024-01-20T08:51:24.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1613 0.20 MB/sec execute 254 sec latency 2307.446 ms 2024-01-20T08:51:25.464 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1617 0.20 MB/sec execute 255 sec latency 2939.419 ms 2024-01-20T08:51:26.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1621 0.20 MB/sec execute 256 sec latency 2762.257 ms 2024-01-20T08:51:27.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1624 0.19 MB/sec execute 257 sec latency 2952.664 ms 2024-01-20T08:51:28.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1627 0.19 MB/sec execute 258 sec latency 2412.461 ms 2024-01-20T08:51:29.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1629 0.19 MB/sec execute 259 sec latency 2323.153 ms 2024-01-20T08:51:30.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1631 0.19 MB/sec execute 260 sec latency 2590.661 ms 2024-01-20T08:51:31.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1637 0.19 MB/sec execute 261 sec latency 2703.601 ms 2024-01-20T08:51:32.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1641 0.19 MB/sec execute 262 sec latency 3217.200 ms 2024-01-20T08:51:33.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1645 0.19 MB/sec execute 263 sec latency 2948.077 ms 2024-01-20T08:51:34.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1648 0.19 MB/sec execute 264 sec latency 2678.374 ms 2024-01-20T08:51:35.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1649 0.19 MB/sec execute 265 sec latency 2680.799 ms 2024-01-20T08:51:36.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1654 0.19 MB/sec execute 266 sec latency 2847.396 ms 2024-01-20T08:51:37.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1659 0.19 MB/sec execute 267 sec latency 3085.305 ms 2024-01-20T08:51:38.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1664 0.19 MB/sec execute 268 sec latency 3503.224 ms 2024-01-20T08:51:39.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1669 0.19 MB/sec execute 269 sec latency 3672.334 ms 2024-01-20T08:51:40.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1672 0.19 MB/sec execute 270 sec latency 3983.712 ms 2024-01-20T08:51:41.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1677 0.19 MB/sec execute 271 sec latency 4209.272 ms 2024-01-20T08:51:42.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1679 0.19 MB/sec execute 272 sec latency 3744.479 ms 2024-01-20T08:51:43.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1681 0.18 MB/sec execute 273 sec latency 4245.055 ms 2024-01-20T08:51:44.348 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:51:44.351 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:51:44.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1686 0.18 MB/sec execute 274 sec latency 4425.431 ms 2024-01-20T08:51:45.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1690 0.18 MB/sec execute 275 sec latency 3759.692 ms 2024-01-20T08:51:46.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1692 0.18 MB/sec execute 276 sec latency 4759.744 ms 2024-01-20T08:51:47.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1695 0.18 MB/sec execute 277 sec latency 5750.487 ms 2024-01-20T08:51:48.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1701 0.18 MB/sec execute 278 sec latency 5886.341 ms 2024-01-20T08:51:49.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1706 0.18 MB/sec execute 279 sec latency 4470.053 ms 2024-01-20T08:51:50.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1707 0.18 MB/sec execute 280 sec latency 3603.498 ms 2024-01-20T08:51:51.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1710 0.18 MB/sec execute 281 sec latency 4296.052 ms 2024-01-20T08:51:52.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1714 0.18 MB/sec execute 282 sec latency 3480.969 ms 2024-01-20T08:51:53.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1719 0.18 MB/sec execute 283 sec latency 3355.911 ms 2024-01-20T08:51:54.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1722 0.18 MB/sec execute 284 sec latency 2736.973 ms 2024-01-20T08:51:55.465 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1724 0.18 MB/sec execute 285 sec latency 3672.903 ms 2024-01-20T08:51:56.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1730 0.18 MB/sec execute 286 sec latency 4400.731 ms 2024-01-20T08:51:57.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1733 0.18 MB/sec execute 287 sec latency 3715.524 ms 2024-01-20T08:51:58.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1737 0.18 MB/sec execute 288 sec latency 4377.076 ms 2024-01-20T08:51:59.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1739 0.18 MB/sec execute 289 sec latency 3957.880 ms 2024-01-20T08:52:00.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1747 0.18 MB/sec execute 290 sec latency 2902.514 ms 2024-01-20T08:52:01.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1750 0.18 MB/sec execute 291 sec latency 3902.546 ms 2024-01-20T08:52:02.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1754 0.17 MB/sec execute 292 sec latency 4902.576 ms 2024-01-20T08:52:03.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1761 0.17 MB/sec execute 293 sec latency 5151.254 ms 2024-01-20T08:52:04.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1763 0.17 MB/sec execute 294 sec latency 4650.885 ms 2024-01-20T08:52:05.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1766 0.17 MB/sec execute 295 sec latency 4352.088 ms 2024-01-20T08:52:06.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1769 0.17 MB/sec execute 296 sec latency 4028.384 ms 2024-01-20T08:52:07.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1772 0.17 MB/sec execute 297 sec latency 3953.941 ms 2024-01-20T08:52:08.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1777 0.17 MB/sec execute 298 sec latency 4471.815 ms 2024-01-20T08:52:09.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1780 0.17 MB/sec execute 299 sec latency 4822.786 ms 2024-01-20T08:52:10.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1784 0.17 MB/sec execute 300 sec latency 5134.850 ms 2024-01-20T08:52:11.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1786 0.17 MB/sec execute 301 sec latency 3526.815 ms 2024-01-20T08:52:12.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1792 0.17 MB/sec execute 302 sec latency 3895.821 ms 2024-01-20T08:52:13.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1801 0.17 MB/sec execute 303 sec latency 2293.733 ms 2024-01-20T08:52:14.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1805 0.17 MB/sec execute 304 sec latency 1944.650 ms 2024-01-20T08:52:14.479 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:52:14.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:52:15.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1806 0.17 MB/sec execute 305 sec latency 2944.698 ms 2024-01-20T08:52:16.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1808 0.17 MB/sec execute 306 sec latency 3505.777 ms 2024-01-20T08:52:17.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1811 0.17 MB/sec execute 307 sec latency 3734.238 ms 2024-01-20T08:52:18.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1816 0.17 MB/sec execute 308 sec latency 2455.196 ms 2024-01-20T08:52:19.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1820 0.17 MB/sec execute 309 sec latency 2322.815 ms 2024-01-20T08:52:20.466 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1822 0.17 MB/sec execute 310 sec latency 2750.253 ms 2024-01-20T08:52:21.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1825 0.17 MB/sec execute 311 sec latency 3102.601 ms 2024-01-20T08:52:22.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1829 0.17 MB/sec execute 312 sec latency 3010.936 ms 2024-01-20T08:52:23.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1830 0.17 MB/sec execute 313 sec latency 1943.500 ms 2024-01-20T08:52:24.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1832 0.16 MB/sec execute 314 sec latency 2358.920 ms 2024-01-20T08:52:25.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1833 0.16 MB/sec execute 315 sec latency 2838.597 ms 2024-01-20T08:52:26.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1841 0.16 MB/sec execute 316 sec latency 3040.925 ms 2024-01-20T08:52:27.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1844 0.16 MB/sec execute 317 sec latency 2824.973 ms 2024-01-20T08:52:28.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1846 0.16 MB/sec execute 318 sec latency 2153.508 ms 2024-01-20T08:52:29.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1848 0.16 MB/sec execute 319 sec latency 2431.212 ms 2024-01-20T08:52:30.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1852 0.16 MB/sec execute 320 sec latency 2241.820 ms 2024-01-20T08:52:31.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1855 0.16 MB/sec execute 321 sec latency 2545.527 ms 2024-01-20T08:52:32.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1856 0.16 MB/sec execute 322 sec latency 2721.221 ms 2024-01-20T08:52:33.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1861 0.16 MB/sec execute 323 sec latency 2738.401 ms 2024-01-20T08:52:34.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1864 0.16 MB/sec execute 324 sec latency 2720.079 ms 2024-01-20T08:52:35.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1866 0.16 MB/sec execute 325 sec latency 2307.813 ms 2024-01-20T08:52:36.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1867 0.16 MB/sec execute 326 sec latency 2611.177 ms 2024-01-20T08:52:37.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1873 0.16 MB/sec execute 327 sec latency 2660.794 ms 2024-01-20T08:52:38.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1876 0.16 MB/sec execute 328 sec latency 1605.469 ms 2024-01-20T08:52:39.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1878 0.16 MB/sec execute 329 sec latency 2156.304 ms 2024-01-20T08:52:40.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1883 0.16 MB/sec execute 330 sec latency 2472.823 ms 2024-01-20T08:52:41.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1890 0.16 MB/sec execute 331 sec latency 2590.190 ms 2024-01-20T08:52:42.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1895 0.16 MB/sec execute 332 sec latency 2517.213 ms 2024-01-20T08:52:43.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1899 0.16 MB/sec execute 333 sec latency 2442.102 ms 2024-01-20T08:52:44.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1902 0.16 MB/sec execute 334 sec latency 2620.860 ms 2024-01-20T08:52:44.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:52:44.554 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:52:45.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1906 0.16 MB/sec execute 335 sec latency 2469.616 ms 2024-01-20T08:52:46.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1908 0.16 MB/sec execute 336 sec latency 2502.812 ms 2024-01-20T08:52:47.467 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1910 0.16 MB/sec execute 337 sec latency 2953.173 ms 2024-01-20T08:52:48.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1915 0.16 MB/sec execute 338 sec latency 3137.087 ms 2024-01-20T08:52:49.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1917 0.16 MB/sec execute 339 sec latency 3382.321 ms 2024-01-20T08:52:50.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1922 0.16 MB/sec execute 340 sec latency 3620.660 ms 2024-01-20T08:52:51.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1926 0.15 MB/sec execute 341 sec latency 3264.406 ms 2024-01-20T08:52:52.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1931 0.15 MB/sec execute 342 sec latency 3125.604 ms 2024-01-20T08:52:53.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1936 0.15 MB/sec execute 343 sec latency 2978.509 ms 2024-01-20T08:52:54.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1938 0.15 MB/sec execute 344 sec latency 3169.160 ms 2024-01-20T08:52:55.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1945 0.15 MB/sec execute 345 sec latency 3287.363 ms 2024-01-20T08:52:56.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1946 0.15 MB/sec execute 346 sec latency 2950.769 ms 2024-01-20T08:52:57.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1951 0.15 MB/sec execute 347 sec latency 3408.991 ms 2024-01-20T08:52:58.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1964 0.15 MB/sec execute 348 sec latency 3084.025 ms 2024-01-20T08:52:59.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1969 0.15 MB/sec execute 349 sec latency 3714.117 ms 2024-01-20T08:53:00.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1972 0.15 MB/sec execute 350 sec latency 3979.103 ms 2024-01-20T08:53:01.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1974 0.15 MB/sec execute 351 sec latency 4248.954 ms 2024-01-20T08:53:02.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1977 0.15 MB/sec execute 352 sec latency 4248.388 ms 2024-01-20T08:53:03.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1978 0.15 MB/sec execute 353 sec latency 5224.585 ms 2024-01-20T08:53:04.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1979 0.15 MB/sec execute 354 sec latency 4493.124 ms 2024-01-20T08:53:05.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1980 0.15 MB/sec execute 355 sec latency 5303.550 ms 2024-01-20T08:53:06.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1983 0.15 MB/sec execute 356 sec latency 5334.192 ms 2024-01-20T08:53:07.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1987 0.15 MB/sec execute 357 sec latency 3423.212 ms 2024-01-20T08:53:08.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1991 0.15 MB/sec execute 358 sec latency 2995.205 ms 2024-01-20T08:53:09.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1994 0.15 MB/sec execute 359 sec latency 2854.971 ms 2024-01-20T08:53:10.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2003 0.15 MB/sec execute 360 sec latency 2966.777 ms 2024-01-20T08:53:11.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2008 0.15 MB/sec execute 361 sec latency 2768.125 ms 2024-01-20T08:53:12.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2011 0.15 MB/sec execute 362 sec latency 3080.779 ms 2024-01-20T08:53:13.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2015 0.15 MB/sec execute 363 sec latency 3093.120 ms 2024-01-20T08:53:14.468 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2017 0.15 MB/sec execute 364 sec latency 2802.729 ms 2024-01-20T08:53:14.684 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:53:14.686 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:53:15.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2019 0.15 MB/sec execute 365 sec latency 3368.232 ms 2024-01-20T08:53:16.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2022 0.15 MB/sec execute 366 sec latency 3953.729 ms 2024-01-20T08:53:17.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2024 0.15 MB/sec execute 367 sec latency 4621.170 ms 2024-01-20T08:53:18.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2030 0.15 MB/sec execute 368 sec latency 5243.000 ms 2024-01-20T08:53:19.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2034 0.15 MB/sec execute 369 sec latency 5270.294 ms 2024-01-20T08:53:20.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2036 0.15 MB/sec execute 370 sec latency 4039.887 ms 2024-01-20T08:53:21.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2038 0.15 MB/sec execute 371 sec latency 2519.771 ms 2024-01-20T08:53:22.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2041 0.15 MB/sec execute 372 sec latency 2528.822 ms 2024-01-20T08:53:23.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2044 0.15 MB/sec execute 373 sec latency 2341.161 ms 2024-01-20T08:53:24.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2049 0.15 MB/sec execute 374 sec latency 2329.330 ms 2024-01-20T08:53:25.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2052 0.15 MB/sec execute 375 sec latency 2881.275 ms 2024-01-20T08:53:26.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2054 0.14 MB/sec execute 376 sec latency 3008.647 ms 2024-01-20T08:53:27.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2055 0.14 MB/sec execute 377 sec latency 3435.724 ms 2024-01-20T08:53:28.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2056 0.14 MB/sec execute 378 sec latency 3550.074 ms 2024-01-20T08:53:29.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2057 0.14 MB/sec execute 379 sec latency 3554.672 ms 2024-01-20T08:53:30.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2058 0.14 MB/sec execute 380 sec latency 2890.353 ms 2024-01-20T08:53:31.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2063 0.14 MB/sec execute 381 sec latency 3182.994 ms 2024-01-20T08:53:32.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2068 0.14 MB/sec execute 382 sec latency 2923.545 ms 2024-01-20T08:53:33.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2070 0.14 MB/sec execute 383 sec latency 2580.194 ms 2024-01-20T08:53:34.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2073 0.14 MB/sec execute 384 sec latency 2332.981 ms 2024-01-20T08:53:35.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2076 0.14 MB/sec execute 385 sec latency 2386.222 ms 2024-01-20T08:53:36.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2077 0.14 MB/sec execute 386 sec latency 2806.767 ms 2024-01-20T08:53:37.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2083 0.14 MB/sec execute 387 sec latency 2725.231 ms 2024-01-20T08:53:38.469 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2087 0.14 MB/sec execute 388 sec latency 3065.047 ms 2024-01-20T08:53:39.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2088 0.14 MB/sec execute 389 sec latency 3315.463 ms 2024-01-20T08:53:40.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2090 0.14 MB/sec execute 390 sec latency 3472.002 ms 2024-01-20T08:53:41.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2093 0.14 MB/sec execute 391 sec latency 3463.584 ms 2024-01-20T08:53:42.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2095 0.14 MB/sec execute 392 sec latency 3209.670 ms 2024-01-20T08:53:43.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2097 0.14 MB/sec execute 393 sec latency 2393.508 ms 2024-01-20T08:53:44.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2100 0.14 MB/sec execute 394 sec latency 2929.701 ms 2024-01-20T08:53:44.788 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:53:44.791 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:53:45.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2104 0.14 MB/sec execute 395 sec latency 3001.282 ms 2024-01-20T08:53:45.591 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:23:30.069 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.595 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:23:30.073 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.602 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:23:30.038 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.604 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:23:30.040 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.605 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:23:30.041 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.607 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:23:30.043 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.738 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:23:30.174 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.743 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:23:30.221 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:45.744 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:23:30.180 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:53:46.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2109 0.14 MB/sec execute 396 sec latency 3281.195 ms 2024-01-20T08:53:47.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2111 0.14 MB/sec execute 397 sec latency 2992.775 ms 2024-01-20T08:53:48.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2116 0.14 MB/sec execute 398 sec latency 3092.716 ms 2024-01-20T08:53:49.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2119 0.14 MB/sec execute 399 sec latency 2921.302 ms 2024-01-20T08:53:50.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2123 0.14 MB/sec execute 400 sec latency 2568.520 ms 2024-01-20T08:53:51.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2127 0.14 MB/sec execute 401 sec latency 3029.304 ms 2024-01-20T08:53:52.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2131 0.14 MB/sec execute 402 sec latency 3143.163 ms 2024-01-20T08:53:53.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2140 0.14 MB/sec execute 403 sec latency 2989.092 ms 2024-01-20T08:53:54.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2142 0.14 MB/sec execute 404 sec latency 3488.497 ms 2024-01-20T08:53:55.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2144 0.14 MB/sec execute 405 sec latency 3126.522 ms 2024-01-20T08:53:56.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2146 0.14 MB/sec execute 406 sec latency 3479.221 ms 2024-01-20T08:53:57.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2148 0.14 MB/sec execute 407 sec latency 4074.619 ms 2024-01-20T08:53:58.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2150 0.14 MB/sec execute 408 sec latency 4482.217 ms 2024-01-20T08:53:59.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2151 0.14 MB/sec execute 409 sec latency 4849.292 ms 2024-01-20T08:54:00.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2153 0.14 MB/sec execute 410 sec latency 4616.131 ms 2024-01-20T08:54:01.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2156 0.13 MB/sec execute 411 sec latency 4934.154 ms 2024-01-20T08:54:02.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2164 0.13 MB/sec execute 412 sec latency 3981.205 ms 2024-01-20T08:54:03.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2173 0.13 MB/sec execute 413 sec latency 2926.646 ms 2024-01-20T08:54:04.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2178 0.14 MB/sec execute 414 sec latency 2630.945 ms 2024-01-20T08:54:05.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2182 0.14 MB/sec execute 415 sec latency 2438.939 ms 2024-01-20T08:54:06.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2184 0.14 MB/sec execute 416 sec latency 2750.323 ms 2024-01-20T08:54:07.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2185 0.14 MB/sec execute 417 sec latency 3104.222 ms 2024-01-20T08:54:08.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2190 0.14 MB/sec execute 418 sec latency 3114.321 ms 2024-01-20T08:54:09.470 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2191 0.14 MB/sec execute 419 sec latency 3040.828 ms 2024-01-20T08:54:10.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2193 0.14 MB/sec execute 420 sec latency 4040.861 ms 2024-01-20T08:54:11.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2198 0.14 MB/sec execute 421 sec latency 4050.293 ms 2024-01-20T08:54:12.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2200 0.14 MB/sec execute 422 sec latency 3980.964 ms 2024-01-20T08:54:13.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2205 0.14 MB/sec execute 423 sec latency 2812.267 ms 2024-01-20T08:54:14.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2210 0.14 MB/sec execute 424 sec latency 2774.973 ms 2024-01-20T08:54:14.918 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:54:14.920 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:54:15.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2212 0.14 MB/sec execute 425 sec latency 3000.815 ms 2024-01-20T08:54:16.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2218 0.14 MB/sec execute 426 sec latency 3307.918 ms 2024-01-20T08:54:17.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2222 0.14 MB/sec execute 427 sec latency 2700.160 ms 2024-01-20T08:54:18.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2228 0.14 MB/sec execute 428 sec latency 3275.828 ms 2024-01-20T08:54:19.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2235 0.14 MB/sec execute 429 sec latency 2795.299 ms 2024-01-20T08:54:20.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2242 0.14 MB/sec execute 430 sec latency 2775.325 ms 2024-01-20T08:54:21.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2246 0.14 MB/sec execute 431 sec latency 2957.697 ms 2024-01-20T08:54:22.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2256 0.14 MB/sec execute 432 sec latency 2941.610 ms 2024-01-20T08:54:23.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2258 0.14 MB/sec execute 433 sec latency 2916.514 ms 2024-01-20T08:54:24.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2262 0.14 MB/sec execute 434 sec latency 3120.894 ms 2024-01-20T08:54:25.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2266 0.14 MB/sec execute 435 sec latency 3130.112 ms 2024-01-20T08:54:26.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2271 0.14 MB/sec execute 436 sec latency 3656.153 ms 2024-01-20T08:54:27.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2274 0.14 MB/sec execute 437 sec latency 4244.721 ms 2024-01-20T08:54:28.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2280 0.14 MB/sec execute 438 sec latency 4430.074 ms 2024-01-20T08:54:29.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2284 0.15 MB/sec execute 439 sec latency 4396.167 ms 2024-01-20T08:54:30.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2287 0.15 MB/sec execute 440 sec latency 4087.905 ms 2024-01-20T08:54:31.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2288 0.15 MB/sec execute 441 sec latency 3347.058 ms 2024-01-20T08:54:32.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2291 0.15 MB/sec execute 442 sec latency 3076.413 ms 2024-01-20T08:54:33.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2292 0.14 MB/sec execute 443 sec latency 3342.774 ms 2024-01-20T08:54:34.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2299 0.14 MB/sec execute 444 sec latency 3179.780 ms 2024-01-20T08:54:35.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2305 0.14 MB/sec execute 445 sec latency 2969.488 ms 2024-01-20T08:54:36.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2309 0.15 MB/sec execute 446 sec latency 2932.726 ms 2024-01-20T08:54:37.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2314 0.15 MB/sec execute 447 sec latency 3470.364 ms 2024-01-20T08:54:38.385 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:24:22.864 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.393 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:24:22.871 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.426 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:24:22.905 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.443 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:24:22.879 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.449 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:24:22.885 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.458 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:24:22.893 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.469 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:24:22.905 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.471 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2316 0.15 MB/sec execute 448 sec latency 3327.728 ms 2024-01-20T08:54:38.507 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:24:22.942 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:38.509 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:24:22.945 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:54:39.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2317 0.15 MB/sec execute 449 sec latency 2357.638 ms 2024-01-20T08:54:40.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2320 0.15 MB/sec execute 450 sec latency 2729.992 ms 2024-01-20T08:54:41.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2321 0.15 MB/sec execute 451 sec latency 2980.992 ms 2024-01-20T08:54:42.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2322 0.15 MB/sec execute 452 sec latency 3429.332 ms 2024-01-20T08:54:43.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2325 0.15 MB/sec execute 453 sec latency 3626.072 ms 2024-01-20T08:54:44.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2329 0.15 MB/sec execute 454 sec latency 3290.527 ms 2024-01-20T08:54:45.249 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:54:45.252 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:54:45.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2331 0.15 MB/sec execute 455 sec latency 2823.111 ms 2024-01-20T08:54:46.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2331 0.15 MB/sec execute 456 sec latency 2857.010 ms 2024-01-20T08:54:47.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2333 0.15 MB/sec execute 457 sec latency 2843.610 ms 2024-01-20T08:54:48.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2334 0.15 MB/sec execute 458 sec latency 2846.425 ms 2024-01-20T08:54:49.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2337 0.15 MB/sec execute 459 sec latency 2670.359 ms 2024-01-20T08:54:50.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2340 0.15 MB/sec execute 460 sec latency 2829.570 ms 2024-01-20T08:54:51.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2342 0.14 MB/sec execute 461 sec latency 3090.389 ms 2024-01-20T08:54:52.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2345 0.15 MB/sec execute 462 sec latency 3297.322 ms 2024-01-20T08:54:53.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2348 0.14 MB/sec execute 463 sec latency 3285.376 ms 2024-01-20T08:54:54.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2350 0.14 MB/sec execute 464 sec latency 2827.559 ms 2024-01-20T08:54:55.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2352 0.14 MB/sec execute 465 sec latency 2580.051 ms 2024-01-20T08:54:56.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2353 0.14 MB/sec execute 466 sec latency 3193.500 ms 2024-01-20T08:54:57.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2355 0.14 MB/sec execute 467 sec latency 2864.027 ms 2024-01-20T08:54:58.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2359 0.14 MB/sec execute 468 sec latency 2875.199 ms 2024-01-20T08:54:59.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2361 0.14 MB/sec execute 469 sec latency 2760.413 ms 2024-01-20T08:55:00.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2363 0.14 MB/sec execute 470 sec latency 2922.599 ms 2024-01-20T08:55:01.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2365 0.14 MB/sec execute 471 sec latency 2889.416 ms 2024-01-20T08:55:02.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2366 0.14 MB/sec execute 472 sec latency 3093.968 ms 2024-01-20T08:55:03.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2367 0.14 MB/sec execute 473 sec latency 3042.650 ms 2024-01-20T08:55:04.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2370 0.14 MB/sec execute 474 sec latency 2829.448 ms 2024-01-20T08:55:05.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2372 0.14 MB/sec execute 475 sec latency 2923.437 ms 2024-01-20T08:55:06.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2375 0.14 MB/sec execute 476 sec latency 3414.003 ms 2024-01-20T08:55:07.472 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2378 0.14 MB/sec execute 477 sec latency 2877.393 ms 2024-01-20T08:55:08.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2380 0.14 MB/sec execute 478 sec latency 3361.190 ms 2024-01-20T08:55:09.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2381 0.14 MB/sec execute 479 sec latency 3047.774 ms 2024-01-20T08:55:10.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2382 0.14 MB/sec execute 480 sec latency 2548.785 ms 2024-01-20T08:55:11.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2385 0.14 MB/sec execute 481 sec latency 2614.253 ms 2024-01-20T08:55:12.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2388 0.14 MB/sec execute 482 sec latency 3018.757 ms 2024-01-20T08:55:13.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2391 0.14 MB/sec execute 483 sec latency 2929.857 ms 2024-01-20T08:55:14.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2392 0.14 MB/sec execute 484 sec latency 2662.268 ms 2024-01-20T08:55:15.303 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:55:15.306 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:55:15.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2394 0.14 MB/sec execute 485 sec latency 3284.380 ms 2024-01-20T08:55:16.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2397 0.14 MB/sec execute 486 sec latency 3187.010 ms 2024-01-20T08:55:17.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2400 0.14 MB/sec execute 487 sec latency 3085.386 ms 2024-01-20T08:55:18.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2404 0.14 MB/sec execute 488 sec latency 3330.909 ms 2024-01-20T08:55:19.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2406 0.14 MB/sec execute 489 sec latency 3479.904 ms 2024-01-20T08:55:20.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2409 0.14 MB/sec execute 490 sec latency 3046.067 ms 2024-01-20T08:55:21.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2411 0.14 MB/sec execute 491 sec latency 3157.156 ms 2024-01-20T08:55:22.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2413 0.14 MB/sec execute 492 sec latency 2982.593 ms 2024-01-20T08:55:23.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2418 0.14 MB/sec execute 493 sec latency 2987.556 ms 2024-01-20T08:55:24.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2422 0.14 MB/sec execute 494 sec latency 2565.578 ms 2024-01-20T08:55:25.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2425 0.14 MB/sec execute 495 sec latency 2109.831 ms 2024-01-20T08:55:26.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2426 0.14 MB/sec execute 496 sec latency 3109.871 ms 2024-01-20T08:55:27.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2428 0.14 MB/sec execute 497 sec latency 4109.903 ms 2024-01-20T08:55:28.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2429 0.14 MB/sec execute 498 sec latency 4292.549 ms 2024-01-20T08:55:29.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2432 0.14 MB/sec execute 499 sec latency 3345.305 ms 2024-01-20T08:55:30.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2436 0.14 MB/sec execute 500 sec latency 2478.948 ms 2024-01-20T08:55:31.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2438 0.14 MB/sec execute 501 sec latency 2849.955 ms 2024-01-20T08:55:32.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2441 0.14 MB/sec execute 502 sec latency 3820.408 ms 2024-01-20T08:55:33.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2443 0.14 MB/sec execute 503 sec latency 3496.554 ms 2024-01-20T08:55:34.473 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2447 0.14 MB/sec execute 504 sec latency 3173.144 ms 2024-01-20T08:55:35.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2448 0.14 MB/sec execute 505 sec latency 2974.671 ms 2024-01-20T08:55:36.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2451 0.14 MB/sec execute 506 sec latency 2950.109 ms 2024-01-20T08:55:37.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2454 0.14 MB/sec execute 507 sec latency 3260.562 ms 2024-01-20T08:55:38.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2457 0.14 MB/sec execute 508 sec latency 3295.250 ms 2024-01-20T08:55:39.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2465 0.14 MB/sec execute 509 sec latency 3378.378 ms 2024-01-20T08:55:40.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2470 0.14 MB/sec execute 510 sec latency 2450.063 ms 2024-01-20T08:55:41.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2471 0.14 MB/sec execute 511 sec latency 2431.482 ms 2024-01-20T08:55:42.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2473 0.14 MB/sec execute 512 sec latency 3431.515 ms 2024-01-20T08:55:43.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2475 0.14 MB/sec execute 513 sec latency 3643.514 ms 2024-01-20T08:55:44.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2479 0.14 MB/sec execute 514 sec latency 2897.212 ms 2024-01-20T08:55:45.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2486 0.14 MB/sec execute 515 sec latency 2788.111 ms 2024-01-20T08:55:45.507 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:55:45.509 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:55:46.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2488 0.14 MB/sec execute 516 sec latency 3173.602 ms 2024-01-20T08:55:47.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2489 0.14 MB/sec execute 517 sec latency 3307.788 ms 2024-01-20T08:55:48.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2491 0.14 MB/sec execute 518 sec latency 3538.928 ms 2024-01-20T08:55:49.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2495 0.14 MB/sec execute 519 sec latency 3335.637 ms 2024-01-20T08:55:50.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2499 0.14 MB/sec execute 520 sec latency 2079.805 ms 2024-01-20T08:55:51.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2500 0.14 MB/sec execute 521 sec latency 2824.690 ms 2024-01-20T08:55:52.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2505 0.15 MB/sec execute 522 sec latency 3123.592 ms 2024-01-20T08:55:53.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2507 0.14 MB/sec execute 523 sec latency 3272.368 ms 2024-01-20T08:55:54.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2517 0.15 MB/sec execute 524 sec latency 3209.514 ms 2024-01-20T08:55:55.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2518 0.15 MB/sec execute 525 sec latency 2562.318 ms 2024-01-20T08:55:56.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2521 0.15 MB/sec execute 526 sec latency 2720.391 ms 2024-01-20T08:55:57.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2524 0.14 MB/sec execute 527 sec latency 3091.787 ms 2024-01-20T08:55:58.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2534 0.15 MB/sec execute 528 sec latency 3202.386 ms 2024-01-20T08:55:58.606 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:25:43.085 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.663 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:25:43.099 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.667 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:25:43.145 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.669 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:25:43.105 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.705 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:25:43.141 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.726 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:25:43.162 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.729 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:25:43.165 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.739 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:25:43.218 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:58.762 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:25:43.198 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:55:59.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2535 0.15 MB/sec execute 529 sec latency 3096.944 ms 2024-01-20T08:56:00.474 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2539 0.14 MB/sec execute 530 sec latency 2734.901 ms 2024-01-20T08:56:01.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2540 0.14 MB/sec execute 531 sec latency 2989.091 ms 2024-01-20T08:56:02.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2544 0.14 MB/sec execute 532 sec latency 2766.033 ms 2024-01-20T08:56:03.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2550 0.14 MB/sec execute 533 sec latency 2762.569 ms 2024-01-20T08:56:04.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2553 0.14 MB/sec execute 534 sec latency 2427.591 ms 2024-01-20T08:56:05.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2558 0.14 MB/sec execute 535 sec latency 2419.611 ms 2024-01-20T08:56:06.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2559 0.14 MB/sec execute 536 sec latency 2479.422 ms 2024-01-20T08:56:07.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2560 0.14 MB/sec execute 537 sec latency 3465.057 ms 2024-01-20T08:56:08.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2562 0.14 MB/sec execute 538 sec latency 3712.225 ms 2024-01-20T08:56:09.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2572 0.14 MB/sec execute 539 sec latency 3670.255 ms 2024-01-20T08:56:10.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2575 0.14 MB/sec execute 540 sec latency 2611.334 ms 2024-01-20T08:56:11.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2576 0.14 MB/sec execute 541 sec latency 2754.029 ms 2024-01-20T08:56:12.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2577 0.14 MB/sec execute 542 sec latency 3333.619 ms 2024-01-20T08:56:13.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2580 0.14 MB/sec execute 543 sec latency 3086.562 ms 2024-01-20T08:56:14.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2583 0.14 MB/sec execute 544 sec latency 2646.956 ms 2024-01-20T08:56:15.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2585 0.14 MB/sec execute 545 sec latency 2819.647 ms 2024-01-20T08:56:15.572 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:56:15.574 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:56:16.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2588 0.14 MB/sec execute 546 sec latency 2991.569 ms 2024-01-20T08:56:17.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2589 0.14 MB/sec execute 547 sec latency 2909.970 ms 2024-01-20T08:56:18.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2590 0.14 MB/sec execute 548 sec latency 3340.936 ms 2024-01-20T08:56:19.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2592 0.14 MB/sec execute 549 sec latency 3433.701 ms 2024-01-20T08:56:20.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2603 0.15 MB/sec execute 550 sec latency 2864.633 ms 2024-01-20T08:56:21.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2605 0.15 MB/sec execute 551 sec latency 3005.075 ms 2024-01-20T08:56:22.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2609 0.15 MB/sec execute 552 sec latency 3211.833 ms 2024-01-20T08:56:23.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2611 0.15 MB/sec execute 553 sec latency 2850.497 ms 2024-01-20T08:56:24.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2616 0.15 MB/sec execute 554 sec latency 2757.169 ms 2024-01-20T08:56:25.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2627 0.15 MB/sec execute 555 sec latency 3134.399 ms 2024-01-20T08:56:26.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2628 0.15 MB/sec execute 556 sec latency 3407.912 ms 2024-01-20T08:56:27.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2629 0.15 MB/sec execute 557 sec latency 2858.869 ms 2024-01-20T08:56:28.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2631 0.15 MB/sec execute 558 sec latency 2852.454 ms 2024-01-20T08:56:29.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2635 0.15 MB/sec execute 559 sec latency 2974.475 ms 2024-01-20T08:56:30.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2638 0.15 MB/sec execute 560 sec latency 2852.610 ms 2024-01-20T08:56:31.475 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2640 0.15 MB/sec execute 561 sec latency 3271.183 ms 2024-01-20T08:56:32.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2641 0.15 MB/sec execute 562 sec latency 3610.810 ms 2024-01-20T08:56:33.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2642 0.15 MB/sec execute 563 sec latency 3350.138 ms 2024-01-20T08:56:34.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2649 0.15 MB/sec execute 564 sec latency 2955.778 ms 2024-01-20T08:56:35.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2665 0.16 MB/sec execute 565 sec latency 2427.946 ms 2024-01-20T08:56:36.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2667 0.16 MB/sec execute 566 sec latency 2902.991 ms 2024-01-20T08:56:37.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2678 0.16 MB/sec execute 567 sec latency 3058.979 ms 2024-01-20T08:56:38.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2680 0.17 MB/sec execute 568 sec latency 3103.621 ms 2024-01-20T08:56:39.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2696 0.17 MB/sec execute 569 sec latency 2891.982 ms 2024-01-20T08:56:40.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2705 0.17 MB/sec execute 570 sec latency 2589.135 ms 2024-01-20T08:56:41.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2708 0.17 MB/sec execute 571 sec latency 2413.071 ms 2024-01-20T08:56:42.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2725 0.18 MB/sec execute 572 sec latency 2522.373 ms 2024-01-20T08:56:43.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2734 0.18 MB/sec execute 573 sec latency 2444.812 ms 2024-01-20T08:56:44.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2744 0.18 MB/sec execute 574 sec latency 2431.104 ms 2024-01-20T08:56:45.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2747 0.18 MB/sec execute 575 sec latency 2830.446 ms 2024-01-20T08:56:45.719 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:56:45.721 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:56:46.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2754 0.18 MB/sec execute 576 sec latency 2764.087 ms 2024-01-20T08:56:47.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2755 0.18 MB/sec execute 577 sec latency 2818.197 ms 2024-01-20T08:56:48.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2765 0.18 MB/sec execute 578 sec latency 2941.714 ms 2024-01-20T08:56:49.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2780 0.19 MB/sec execute 579 sec latency 3245.650 ms 2024-01-20T08:56:50.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2784 0.19 MB/sec execute 580 sec latency 3198.550 ms 2024-01-20T08:56:51.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2787 0.19 MB/sec execute 581 sec latency 2305.760 ms 2024-01-20T08:56:52.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2795 0.19 MB/sec execute 582 sec latency 1495.715 ms 2024-01-20T08:56:53.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2798 0.19 MB/sec execute 583 sec latency 2495.745 ms 2024-01-20T08:56:54.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2799 0.19 MB/sec execute 584 sec latency 3377.605 ms 2024-01-20T08:56:55.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2803 0.19 MB/sec execute 585 sec latency 2935.314 ms 2024-01-20T08:56:56.476 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2826 0.19 MB/sec execute 586 sec latency 2605.271 ms 2024-01-20T08:56:57.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2837 0.19 MB/sec execute 587 sec latency 1783.886 ms 2024-01-20T08:56:58.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2846 0.19 MB/sec execute 588 sec latency 2783.913 ms 2024-01-20T08:56:59.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2847 0.19 MB/sec execute 589 sec latency 3313.854 ms 2024-01-20T08:57:00.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2850 0.19 MB/sec execute 590 sec latency 2810.302 ms 2024-01-20T08:57:01.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2854 0.19 MB/sec execute 591 sec latency 2688.325 ms 2024-01-20T08:57:02.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2858 0.19 MB/sec execute 592 sec latency 3001.602 ms 2024-01-20T08:57:03.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2880 0.19 MB/sec execute 593 sec latency 2917.912 ms 2024-01-20T08:57:04.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2884 0.19 MB/sec execute 594 sec latency 3204.345 ms 2024-01-20T08:57:05.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2886 0.19 MB/sec execute 595 sec latency 2341.714 ms 2024-01-20T08:57:06.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2892 0.20 MB/sec execute 596 sec latency 2947.894 ms 2024-01-20T08:57:07.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2893 0.20 MB/sec execute 597 sec latency 2366.155 ms 2024-01-20T08:57:08.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2897 0.20 MB/sec execute 598 sec latency 3366.206 ms 2024-01-20T08:57:09.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2902 0.20 MB/sec execute 599 sec latency 3608.738 ms 2024-01-20T08:57:10.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 600 sec 2024-01-20T08:57:11.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 601 sec 2024-01-20T08:57:12.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 602 sec 2024-01-20T08:57:13.173 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:26:57.652 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.177 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:26:57.655 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.182 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:26:57.618 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.184 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:26:57.620 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.185 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:26:57.621 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.188 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:26:57.624 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.225 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:26:57.704 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.227 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:26:57.663 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.229 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:26:57.665 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T08:57:13.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 603 sec 2024-01-20T08:57:14.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 604 sec 2024-01-20T08:57:15.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 605 sec 2024-01-20T08:57:15.868 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:57:15.870 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:57:16.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 606 sec 2024-01-20T08:57:17.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 607 sec 2024-01-20T08:57:18.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 608 sec 2024-01-20T08:57:19.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 609 sec 2024-01-20T08:57:20.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 610 sec 2024-01-20T08:57:21.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 611 sec 2024-01-20T08:57:22.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 612 sec 2024-01-20T08:57:23.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 613 sec 2024-01-20T08:57:24.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 614 sec 2024-01-20T08:57:25.477 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 615 sec 2024-01-20T08:57:26.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 616 sec 2024-01-20T08:57:27.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 617 sec 2024-01-20T08:57:28.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 618 sec 2024-01-20T08:57:29.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 619 sec 2024-01-20T08:57:30.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 620 sec 2024-01-20T08:57:31.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 621 sec 2024-01-20T08:57:32.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 622 sec 2024-01-20T08:57:33.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 623 sec 2024-01-20T08:57:34.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 624 sec 2024-01-20T08:57:35.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 625 sec 2024-01-20T08:57:36.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 626 sec 2024-01-20T08:57:37.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 627 sec 2024-01-20T08:57:38.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 628 sec 2024-01-20T08:57:39.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 629 sec 2024-01-20T08:57:40.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 630 sec 2024-01-20T08:57:41.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 631 sec 2024-01-20T08:57:42.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 632 sec 2024-01-20T08:57:43.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 633 sec 2024-01-20T08:57:44.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 634 sec 2024-01-20T08:57:45.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 635 sec 2024-01-20T08:57:45.931 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:57:45.933 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:57:46.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 636 sec 2024-01-20T08:57:47.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 637 sec 2024-01-20T08:57:48.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 638 sec 2024-01-20T08:57:49.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 639 sec 2024-01-20T08:57:50.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 640 sec 2024-01-20T08:57:51.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 641 sec 2024-01-20T08:57:52.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 642 sec 2024-01-20T08:57:53.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 643 sec 2024-01-20T08:57:54.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 644 sec 2024-01-20T08:57:55.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 645 sec 2024-01-20T08:57:56.478 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 646 sec 2024-01-20T08:57:57.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 647 sec 2024-01-20T08:57:58.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 648 sec 2024-01-20T08:57:59.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 649 sec 2024-01-20T08:58:00.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 650 sec 2024-01-20T08:58:01.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 651 sec 2024-01-20T08:58:02.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 652 sec 2024-01-20T08:58:03.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 653 sec 2024-01-20T08:58:04.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 654 sec 2024-01-20T08:58:05.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 655 sec 2024-01-20T08:58:06.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 656 sec 2024-01-20T08:58:07.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 657 sec 2024-01-20T08:58:08.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 658 sec 2024-01-20T08:58:09.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 659 sec 2024-01-20T08:58:10.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 660 sec 2024-01-20T08:58:11.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 661 sec 2024-01-20T08:58:12.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 662 sec 2024-01-20T08:58:13.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 663 sec 2024-01-20T08:58:14.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 664 sec 2024-01-20T08:58:15.479 INFO:tasks.workunit.client.0.smithi087.stdout: 10 cleanup 665 sec 2024-01-20T08:58:15.996 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:58:15.999 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:58:16.479 INFO:tasks.workunit.client.0.smithi087.stdout: 9 cleanup 666 sec 2024-01-20T08:58:17.480 INFO:tasks.workunit.client.0.smithi087.stdout: 8 cleanup 667 sec 2024-01-20T08:58:18.479 INFO:tasks.workunit.client.0.smithi087.stdout: 8 cleanup 668 sec 2024-01-20T08:58:19.479 INFO:tasks.workunit.client.0.smithi087.stdout: 8 cleanup 669 sec 2024-01-20T08:58:20.479 INFO:tasks.workunit.client.0.smithi087.stdout: 7 cleanup 670 sec 2024-01-20T08:58:21.479 INFO:tasks.workunit.client.0.smithi087.stdout: 5 cleanup 671 sec 2024-01-20T08:58:22.479 INFO:tasks.workunit.client.0.smithi087.stdout: 4 cleanup 672 sec 2024-01-20T08:58:23.479 INFO:tasks.workunit.client.0.smithi087.stdout: 3 cleanup 673 sec 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: 0 cleanup 674 sec 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: Operation Count AvgLat MaxLat 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: ---------------------------------------- 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: NTCreateX 3705 308.141 1962.961 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: Close 3125 3.070 58.245 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: Rename 185 911.560 1649.005 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: Unlink 420 295.609 783.714 2024-01-20T08:58:24.220 INFO:tasks.workunit.client.0.smithi087.stdout: Qpathinfo 3656 202.638 1674.890 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: Qfileinfo 759 6.214 44.536 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: Qfsinfo 480 23.215 142.558 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: Sfileinfo 473 40.115 155.693 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: Find 1222 2994.572 5886.332 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: WriteX 2142 14.380 800.126 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: ReadX 5942 0.082 26.722 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: LockX 20 168.790 310.256 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: UnlockX 20 144.475 283.314 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: Flush 312 320.138 883.689 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-01-20T08:58:24.221 INFO:tasks.workunit.client.0.smithi087.stdout:Throughput 0.197357 MB/sec 10 clients 10 procs max_latency=5886.341 ms 2024-01-20T08:58:24.221 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-01-20T08:58:24.222 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-20T08:58:45.622 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-01-20T08:58:45.622 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-01-20T08:58:45.887 DEBUG:teuthology.parallel:result is None 2024-01-20T08:58:45.887 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-01-20T08:58:46.126 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:58:46.128 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:58:46.133 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-01-20T08:58:46.134 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-01-20T08:58:46.143 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-01-20T08:58:46.144 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:58:46.144 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-20T08:58:46.189 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:58:46.189 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-20T08:58:46.244 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:58:46.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.388+0000 7f2e7c557640 1 Processor -- start 2024-01-20T08:58:46.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.388+0000 7f2e7c557640 1 -- start start 2024-01-20T08:58:46.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.389+0000 7f2e7c557640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e740c8380 0x7f2e74159610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.389+0000 7f2e7c557640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e74159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.389+0000 7f2e7c557640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7406fd40 con 0x7f2e741501c0 2024-01-20T08:58:46.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.389+0000 7f2e7c557640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740c2710 con 0x7f2e740c8380 2024-01-20T08:58:46.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.389+0000 7f2e7c557640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74070000 con 0x7f2e7406f800 2024-01-20T08:58:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.390+0000 7f2e79af2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e740c8380 0x7f2e74159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.391+0000 7f2e79af2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e740c8380 0x7f2e74159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49742/0 (socket says 172.21.15.87:49742) 2024-01-20T08:58:46.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.391+0000 7f2e79af2640 1 -- 172.21.15.87:0/2620071001 learned_addr learned my addr 172.21.15.87:0/2620071001 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.432+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048659989 0 0) 0x7f2e7406fd40 con 0x7f2e741501c0 2024-01-20T08:58:46.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.432+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e60003520 con 0x7f2e741501c0 2024-01-20T08:58:46.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.434+0000 7f2e7aaf4640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e74159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682974760 0 0) 0x7f2e60003520 con 0x7f2e741501c0 2024-01-20T08:58:46.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 msgr2=0x7f2e74159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e74159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 >> v2:172.21.15.116:3300/0 conn(0x7f2e740c8380 msgr2=0x7f2e74159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 --2- 172.21.15.87:0/2620071001 >> v2:172.21.15.116:3300/0 conn(0x7f2e740c8380 0x7f2e74159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740702e0 con 0x7f2e741501c0 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4017552009 0 0) 0x7f2e70001670 con 0x7f2e741501c0 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 >> v1:172.21.15.87:6789/0 conn(0x7f2e741501c0 legacy=0x7f2e740c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e60003970 0x7f2e60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e60004d00 0x7f2e60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c2710 con 0x7f2e60003970 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74070000 con 0x7f2e60004d00 2024-01-20T08:58:46.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.436+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6000ba30 con 0x7f2e7406f800 2024-01-20T08:58:46.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.438+0000 7f2e79af2640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.439+0000 7f2e7a2f3640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e60003970 0x7f2e60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.441+0000 7f2e7aaf4640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e60004d00 0x7f2e60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.441+0000 7f2e79af2640 1 -- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e60004d00 msgr2=0x7f2e60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:46.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.441+0000 7f2e79af2640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e60004d00 0x7f2e60005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.441+0000 7f2e79af2640 1 -- 172.21.15.87:0/2620071001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e60003970 msgr2=0x7f2e60003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:46.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.441+0000 7f2e79af2640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e60003970 0x7f2e60003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.441+0000 7f2e79af2640 1 -- 172.21.15.87:0/2620071001 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e60003520 con 0x7f2e7406f800 2024-01-20T08:58:46.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.443+0000 7f2e79af2640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e600047c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e640098a0 tx=0x7f2e600057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed68d7394674ec2 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:46.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.447+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64030660 con 0x7f2e7406f800 2024-01-20T08:58:46.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.447+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e640307c0 con 0x7f2e7406f800 2024-01-20T08:58:46.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.447+0000 7f2e792f1640 1 -- 172.21.15.87:0/2620071001 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64030b50 con 0x7f2e7406f800 2024-01-20T08:58:46.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7c557640 1 -- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 msgr2=0x7f2e600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:46.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7c557640 1 --2- 172.21.15.87:0/2620071001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7406f800 0x7f2e600047c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e640098a0 tx=0x7f2e600057f0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7aaf4640 1 -- 172.21.15.87:0/2620071001 reap_dead start 2024-01-20T08:58:46.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7c557640 1 -- 172.21.15.87:0/2620071001 shutdown_connections 2024-01-20T08:58:46.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7c557640 1 -- 172.21.15.87:0/2620071001 >> 172.21.15.87:0/2620071001 conn(0x7f2e740cbe60 msgr2=0x7f2e740cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:46.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7c557640 1 -- 172.21.15.87:0/2620071001 shutdown_connections 2024-01-20T08:58:46.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.448+0000 7f2e7c557640 1 -- 172.21.15.87:0/2620071001 wait complete. 2024-01-20T08:58:46.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.449+0000 7f2e7c557640 1 Processor -- start 2024-01-20T08:58:46.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.449+0000 7f2e7c557640 1 -- start start 2024-01-20T08:58:46.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7c557640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e7406f800 0x7f2e740caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7c557640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7414de40 0x7f2e741ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7c557640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7414c6f0 0x7f2e741efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7c557640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7414bfc0 con 0x7f2e7414c6f0 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7c557640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7414f9e0 con 0x7f2e7406f800 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7c557640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7414c210 con 0x7f2e7414de40 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7aaf4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7414c6f0 0x7f2e741efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e7a2f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e7406f800 0x7f2e740caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.450+0000 7f2e79af2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7414de40 0x7f2e741ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.451+0000 7f2e7a2f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e7406f800 0x7f2e740caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49764/0 (socket says 172.21.15.87:49764) 2024-01-20T08:58:46.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.451+0000 7f2e7a2f3640 1 -- 172.21.15.87:0/1706783659 learned_addr learned my addr 172.21.15.87:0/1706783659 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:46.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.454+0000 7f2e7aaf4640 1 -- 172.21.15.87:0/1706783659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7414de40 msgr2=0x7f2e741ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:46.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.454+0000 7f2e7aaf4640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7414de40 0x7f2e741ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.454+0000 7f2e7aaf4640 1 -- 172.21.15.87:0/1706783659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e7406f800 msgr2=0x7f2e740caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:46.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.454+0000 7f2e7aaf4640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e7406f800 0x7f2e740caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.454+0000 7f2e7aaf4640 1 -- 172.21.15.87:0/1706783659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7414edd0 con 0x7f2e7414c6f0 2024-01-20T08:58:46.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.456+0000 7f2e7aaf4640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7414c6f0 0x7f2e741efb50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e740c99b0 tx=0x7f2e6c009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:46.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.460+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c007cf0 con 0x7f2e7414c6f0 2024-01-20T08:58:46.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.460+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c00f050 con 0x7f2e7414c6f0 2024-01-20T08:58:46.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.460+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7414f0b0 con 0x7f2e7414c6f0 2024-01-20T08:58:46.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.460+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c014650 con 0x7f2e7414c6f0 2024-01-20T08:58:46.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.460+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7414f670 con 0x7f2e7414c6f0 2024-01-20T08:58:46.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.461+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e74074290 con 0x7f2e7414c6f0 2024-01-20T08:58:46.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.479+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2e6c0147b0 con 0x7f2e7414c6f0 2024-01-20T08:58:46.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.480+0000 7f2e6affd640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2e5806aed0 0x7f2e5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:46.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.480+0000 7f2e7a2f3640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2e5806aed0 0x7f2e5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:46.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.480+0000 7f2e7a2f3640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2e5806aed0 0x7f2e5806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e70001820 tx=0x7f2e70009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:46.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.483+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f2e6c0592d0 con 0x7f2e7414c6f0 2024-01-20T08:58:46.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.720+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2e6c055610 con 0x7f2e7414c6f0 2024-01-20T08:58:46.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.897+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e74156f30 con 0x7f2e7414c6f0 2024-01-20T08:58:46.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.934+0000 7f2e6affd640 1 -- 172.21.15.87:0/1706783659 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7f2e6c058c30 con 0x7f2e7414c6f0 2024-01-20T08:58:46.936 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:58:46.936 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:58:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2e5806aed0 msgr2=0x7f2e5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2e5806aed0 0x7f2e5806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e70001820 tx=0x7f2e70009040 comp rx=0 tx=0).stop 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7414c6f0 msgr2=0x7f2e741efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7414c6f0 0x7f2e741efb50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e740c99b0 tx=0x7f2e6c009d50 comp rx=0 tx=0).stop 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 shutdown_connections 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f2e5806aed0 0x7f2e5806d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7414c6f0 0x7f2e741efb50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e7414de40 0x7f2e741ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 --2- 172.21.15.87:0/1706783659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e7406f800 0x7f2e740caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 >> 172.21.15.87:0/1706783659 conn(0x7f2e740cbe60 msgr2=0x7f2e7406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 shutdown_connections 2024-01-20T08:58:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:46.936+0000 7f2e7c557640 1 -- 172.21.15.87:0/1706783659 wait complete. 2024-01-20T08:58:46.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-20T08:31:06.500533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-20T08:31:08.175669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-20T08:31:13.903374+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-20T08:58:46.950 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:58:47.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.053+0000 7f08c5c41640 1 Processor -- start 2024-01-20T08:58:47.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.053+0000 7f08c5c41640 1 -- start start 2024-01-20T08:58:47.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08c5c41640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f08c01505f0 0x7f08c00c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08c5c41640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08c00c71c0 0x7f08c01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08c5c41640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00c7610 con 0x7f08c00c7dc0 2024-01-20T08:58:47.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08c5c41640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c0071f20 con 0x7f08c00c71c0 2024-01-20T08:58:47.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08c5c41640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c0071c40 con 0x7f08c01505f0 2024-01-20T08:58:47.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08beffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08c00c71c0 0x7f08c01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.054+0000 7f08bf7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f08c01505f0 0x7f08c00c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.055+0000 7f08beffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08c00c71c0 0x7f08c01599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59688/0 (socket says 172.21.15.87:59688) 2024-01-20T08:58:47.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.055+0000 7f08beffd640 1 -- 172.21.15.87:0/732407803 learned_addr learned my addr 172.21.15.87:0/732407803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:47.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.057+0000 7f08bf7fe640 1 -- 172.21.15.87:0/732407803 >> v2:172.21.15.87:3300/0 conn(0x7f08c00c71c0 msgr2=0x7f08c01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:47.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.057+0000 7f08bf7fe640 1 --2- 172.21.15.87:0/732407803 >> v2:172.21.15.87:3300/0 conn(0x7f08c00c71c0 0x7f08c01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.057+0000 7f08bf7fe640 1 -- 172.21.15.87:0/732407803 >> v1:172.21.15.87:6789/0 conn(0x7f08c00c7dc0 legacy=0x7f08c006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.057+0000 7f08bf7fe640 1 -- 172.21.15.87:0/732407803 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00727b0 con 0x7f08c01505f0 2024-01-20T08:58:47.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.059+0000 7f08bf7fe640 1 --2- 172.21.15.87:0/732407803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c00c5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cbe50 tx=0x7f08b002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf3e60523360ee3 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:47.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.063+0000 7f08be7fc640 1 -- 172.21.15.87:0/732407803 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0031830 con 0x7f08c01505f0 2024-01-20T08:58:47.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.063+0000 7f08be7fc640 1 -- 172.21.15.87:0/732407803 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0031990 con 0x7f08c01505f0 2024-01-20T08:58:47.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.063+0000 7f08be7fc640 1 -- 172.21.15.87:0/732407803 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0031d20 con 0x7f08c01505f0 2024-01-20T08:58:47.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 -- 172.21.15.87:0/732407803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 msgr2=0x7f08c00c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 --2- 172.21.15.87:0/732407803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c00c5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cbe50 tx=0x7f08b002b8d0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 -- 172.21.15.87:0/732407803 shutdown_connections 2024-01-20T08:58:47.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 --2- 172.21.15.87:0/732407803 >> v2:172.21.15.87:3300/0 conn(0x7f08c00c71c0 0x7f08c01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 --2- 172.21.15.87:0/732407803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c00c5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 -- 172.21.15.87:0/732407803 >> 172.21.15.87:0/732407803 conn(0x7f08c006a9a0 msgr2=0x7f08c006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 -- 172.21.15.87:0/732407803 shutdown_connections 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.064+0000 7f08c5c41640 1 -- 172.21.15.87:0/732407803 wait complete. 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.065+0000 7f08c5c41640 1 Processor -- start 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.065+0000 7f08c5c41640 1 -- start start 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.066+0000 7f08c5c41640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00c71c0 0x7f08c0152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.066+0000 7f08c5c41640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c00c7dc0 0x7f08c01f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.066+0000 7f08c5c41640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c0162e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.066+0000 7f08c5c41640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c013e4f0 con 0x7f08c00c71c0 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.066+0000 7f08c5c41640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0150cb0 con 0x7f08c01505f0 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.066+0000 7f08c5c41640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0150f90 con 0x7f08c00c7dc0 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.067+0000 7f08bffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c0162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.067+0000 7f08bf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00c71c0 0x7f08c0152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.067+0000 7f08beffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c00c7dc0 0x7f08c01f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.067+0000 7f08bffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c0162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49782/0 (socket says 172.21.15.87:49782) 2024-01-20T08:58:47.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.067+0000 7f08bffff640 1 -- 172.21.15.87:0/924147020 learned_addr learned my addr 172.21.15.87:0/924147020 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:47.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.070+0000 7f08bf7fe640 1 -- 172.21.15.87:0/924147020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c00c7dc0 msgr2=0x7f08c01f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.070+0000 7f08bf7fe640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c00c7dc0 0x7f08c01f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.070+0000 7f08bf7fe640 1 -- 172.21.15.87:0/924147020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 msgr2=0x7f08c0162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.070+0000 7f08bf7fe640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c0162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.070+0000 7f08bf7fe640 1 -- 172.21.15.87:0/924147020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0009400 con 0x7f08c00c71c0 2024-01-20T08:58:47.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.073+0000 7f08bf7fe640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00c71c0 0x7f08c0152030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f08b0033000 tx=0x7f08b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:47.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.077+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b003a6f0 con 0x7f08c00c71c0 2024-01-20T08:58:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.077+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b003a870 con 0x7f08c00c71c0 2024-01-20T08:58:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.077+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b003ac20 con 0x7f08c00c71c0 2024-01-20T08:58:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.077+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c0151270 con 0x7f08c00c71c0 2024-01-20T08:58:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.077+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c0151830 con 0x7f08c00c71c0 2024-01-20T08:58:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.078+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c014f360 con 0x7f08c00c71c0 2024-01-20T08:58:47.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.097+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f08b0030380 con 0x7f08c00c71c0 2024-01-20T08:58:47.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.098+0000 7f08a7fff640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08ac06af00 0x7f08ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.098+0000 7f08beffd640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08ac06af00 0x7f08ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.098+0000 7f08beffd640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08ac06af00 0x7f08ac06d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009b30 tx=0x7f08b4009240 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:47.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.355+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f08b007a1a0 con 0x7f08c00c71c0 2024-01-20T08:58:47.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.355+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f08b00764e0 con 0x7f08c00c71c0 2024-01-20T08:58:47.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.541+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08c00c5280 con 0x7f08c00c71c0 2024-01-20T08:58:47.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.569+0000 7f08a7fff640 1 -- 172.21.15.87:0/924147020 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08b0048030 con 0x7f08c00c71c0 2024-01-20T08:58:47.571 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08ac06af00 msgr2=0x7f08ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08ac06af00 0x7f08ac06d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009b30 tx=0x7f08b4009240 comp rx=0 tx=0).stop 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00c71c0 msgr2=0x7f08c0152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00c71c0 0x7f08c0152030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f08b0033000 tx=0x7f08b00046c0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 shutdown_connections 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f08ac06af00 0x7f08ac06d3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c01505f0 0x7f08c0162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c00c7dc0 0x7f08c01f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 --2- 172.21.15.87:0/924147020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00c71c0 0x7f08c0152030 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.572+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 >> 172.21.15.87:0/924147020 conn(0x7f08c006a9a0 msgr2=0x7f08c00c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.573+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 shutdown_connections 2024-01-20T08:58:47.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.573+0000 7f08c5c41640 1 -- 172.21.15.87:0/924147020 wait complete. 2024-01-20T08:58:47.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-20T08:58:47.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.688+0000 7f9984341640 1 Processor -- start 2024-01-20T08:58:47.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.688+0000 7f9984341640 1 -- start start 2024-01-20T08:58:47.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.689+0000 7f9984341640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f997c15a930 0x7f997c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.689+0000 7f9984341640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c151b10 con 0x7f997c0c8f80 2024-01-20T08:58:47.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.689+0000 7f9984341640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c0c2300 con 0x7f997c151730 2024-01-20T08:58:47.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.689+0000 7f9984341640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c15b2c0 con 0x7f997c15a930 2024-01-20T08:58:47.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.690+0000 7f99818dc640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f997c0c8f80 0x7f997c0c9a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59050/0 (socket says 172.21.15.87:59050) 2024-01-20T08:58:47.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.690+0000 7f99818dc640 1 -- 172.21.15.87:0/2359583340 learned_addr learned my addr 172.21.15.87:0/2359583340 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:47.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.690+0000 7f99828de640 1 --2- 172.21.15.87:0/2359583340 >> v2:172.21.15.116:3300/0 conn(0x7f997c15a930 0x7f997c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.692+0000 7f99810db640 1 -- 172.21.15.87:0/2359583340 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388321891 0 0) 0x7f997c151b10 con 0x7f997c0c8f80 2024-01-20T08:58:47.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.692+0000 7f99810db640 1 -- 172.21.15.87:0/2359583340 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9964003520 con 0x7f997c0c8f80 2024-01-20T08:58:47.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.693+0000 7f99828de640 1 -- 172.21.15.87:0/2359583340 >> v1:172.21.15.116:6789/0 conn(0x7f997c151730 legacy=0x7f997c0c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.693+0000 7f99828de640 1 -- 172.21.15.87:0/2359583340 >> v1:172.21.15.87:6789/0 conn(0x7f997c0c8f80 legacy=0x7f997c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.693+0000 7f99828de640 1 -- 172.21.15.87:0/2359583340 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c15b7a0 con 0x7f997c15a930 2024-01-20T08:58:47.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.697+0000 7f99828de640 1 --2- 172.21.15.87:0/2359583340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c15ad80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99740097a0 tx=0x7f997402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a8e3e31b706507 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.702+0000 7f99810db640 1 -- 172.21.15.87:0/2359583340 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997402f810 con 0x7f997c15a930 2024-01-20T08:58:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.702+0000 7f99810db640 1 -- 172.21.15.87:0/2359583340 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997402f970 con 0x7f997c15a930 2024-01-20T08:58:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.702+0000 7f99810db640 1 -- 172.21.15.87:0/2359583340 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997402fd00 con 0x7f997c15a930 2024-01-20T08:58:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.702+0000 7f9984341640 1 -- 172.21.15.87:0/2359583340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 msgr2=0x7f997c15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.702+0000 7f9984341640 1 --2- 172.21.15.87:0/2359583340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c15ad80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99740097a0 tx=0x7f997402b9f0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.703+0000 7f9984341640 1 -- 172.21.15.87:0/2359583340 shutdown_connections 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.703+0000 7f9984341640 1 --2- 172.21.15.87:0/2359583340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c15ad80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.703+0000 7f9984341640 1 -- 172.21.15.87:0/2359583340 >> 172.21.15.87:0/2359583340 conn(0x7f997c070de0 msgr2=0x7f997c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.703+0000 7f9984341640 1 -- 172.21.15.87:0/2359583340 shutdown_connections 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.703+0000 7f9984341640 1 -- 172.21.15.87:0/2359583340 wait complete. 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 Processor -- start 2024-01-20T08:58:47.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 -- start start 2024-01-20T08:58:47.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f997c0c8f80 0x7f997c1ed840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 0x7f997c0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0c6310 con 0x7f997c151730 2024-01-20T08:58:47.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0c6560 con 0x7f997c15a930 2024-01-20T08:58:47.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.704+0000 7f9984341640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0c6840 con 0x7f997c0c8f80 2024-01-20T08:58:47.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.705+0000 7f99818dc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 0x7f997c0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.705+0000 7f99828de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.705+0000 7f99820dd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f997c0c8f80 0x7f997c1ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.705+0000 7f99818dc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 0x7f997c0c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59712/0 (socket says 172.21.15.87:59712) 2024-01-20T08:58:47.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.705+0000 7f99818dc640 1 -- 172.21.15.87:0/667178965 learned_addr learned my addr 172.21.15.87:0/667178965 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:47.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.709+0000 7f99818dc640 1 -- 172.21.15.87:0/667178965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f997c0c8f80 msgr2=0x7f997c1ed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.709+0000 7f99818dc640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f997c0c8f80 0x7f997c1ed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.709+0000 7f99818dc640 1 -- 172.21.15.87:0/667178965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 msgr2=0x7f997c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:47.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.709+0000 7f99818dc640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:47.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.709+0000 7f99818dc640 1 -- 172.21.15.87:0/667178965 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974009400 con 0x7f997c151730 2024-01-20T08:58:47.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.711+0000 7f99818dc640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 0x7f997c0c59d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f997000c500 tx=0x7f997000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:47.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.714+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970005ce0 con 0x7f997c151730 2024-01-20T08:58:47.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.714+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9970005e40 con 0x7f997c151730 2024-01-20T08:58:47.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.714+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c0c6b20 con 0x7f997c151730 2024-01-20T08:58:47.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.714+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970007330 con 0x7f997c151730 2024-01-20T08:58:47.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.715+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c0c70e0 con 0x7f997c151730 2024-01-20T08:58:47.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.715+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c1512e0 con 0x7f997c151730 2024-01-20T08:58:47.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.958+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9970007490 con 0x7f997c151730 2024-01-20T08:58:47.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.959+0000 7f997affd640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f996806af50 0x7f996806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:47.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.959+0000 7f99820dd640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f996806af50 0x7f996806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:47.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.959+0000 7f99820dd640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f996806af50 0x7f996806d410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f996c001f10 tx=0x7f996c009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:47.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.973+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9970007740 con 0x7f997c151730 2024-01-20T08:58:47.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:47.973+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f99700540b0 con 0x7f997c151730 2024-01-20T08:58:48.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.148+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f997c074290 con 0x7f997c151730 2024-01-20T08:58:48.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.153+0000 7f997affd640 1 -- 172.21.15.87:0/667178965 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99700170a0 con 0x7f997c151730 2024-01-20T08:58:48.154 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f996806af50 msgr2=0x7f996806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f996806af50 0x7f996806d410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f996c001f10 tx=0x7f996c009040 comp rx=0 tx=0).stop 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 msgr2=0x7f997c0c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 0x7f997c0c59d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f997000c500 tx=0x7f997000c9c0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 shutdown_connections 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f996806af50 0x7f996806d410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f997c15a930 0x7f997c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c151730 0x7f997c0c59d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 --2- 172.21.15.87:0/667178965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f997c0c8f80 0x7f997c1ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 >> 172.21.15.87:0/667178965 conn(0x7f997c070de0 msgr2=0x7f997c0cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 shutdown_connections 2024-01-20T08:58:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.155+0000 7f9984341640 1 -- 172.21.15.87:0/667178965 wait complete. 2024-01-20T08:58:48.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-20T08:58:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.269+0000 7f005274c640 1 Processor -- start 2024-01-20T08:58:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.269+0000 7f005274c640 1 -- start start 2024-01-20T08:58:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.270+0000 7f005274c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f004c153600 0x7f004c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.270+0000 7f005274c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c0c4fa0 0x7f004c159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.270+0000 7f005274c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c06c560 con 0x7f004c072740 2024-01-20T08:58:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.270+0000 7f005274c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c15dc30 con 0x7f004c153600 2024-01-20T08:58:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.270+0000 7f005274c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c06c840 con 0x7f004c0c4fa0 2024-01-20T08:58:48.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.271+0000 7f004b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c0c4fa0 0x7f004c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.271+0000 7f0050ce9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f004c072740 0x7f004c159b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:48144/0 (socket says 172.21.15.87:48144) 2024-01-20T08:58:48.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.271+0000 7f0050ce9640 1 -- 172.21.15.87:0/3772205408 learned_addr learned my addr 172.21.15.87:0/3772205408 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:48.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.271+0000 7f004bfff640 1 --2- 172.21.15.87:0/3772205408 >> v2:172.21.15.116:3300/0 conn(0x7f004c153600 0x7f004c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.274+0000 7f004b7fe640 1 -- 172.21.15.87:0/3772205408 >> v2:172.21.15.116:3300/0 conn(0x7f004c153600 msgr2=0x7f004c0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:48.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.274+0000 7f004b7fe640 1 --2- 172.21.15.87:0/3772205408 >> v2:172.21.15.116:3300/0 conn(0x7f004c153600 0x7f004c0c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.274+0000 7f004b7fe640 1 -- 172.21.15.87:0/3772205408 >> v1:172.21.15.116:6789/0 conn(0x7f004c072740 legacy=0x7f004c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.274+0000 7f004b7fe640 1 -- 172.21.15.87:0/3772205408 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0c3f80 con 0x7f004c0c4fa0 2024-01-20T08:58:48.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.276+0000 7f004b7fe640 1 --2- 172.21.15.87:0/3772205408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c0c4fa0 0x7f004c159630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00400097a0 tx=0x7f004002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af714d4b315079d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:48.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.279+0000 7f004affd640 1 -- 172.21.15.87:0/3772205408 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040031830 con 0x7f004c0c4fa0 2024-01-20T08:58:48.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f004affd640 1 -- 172.21.15.87:0/3772205408 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040031990 con 0x7f004c0c4fa0 2024-01-20T08:58:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f004affd640 1 -- 172.21.15.87:0/3772205408 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040031d20 con 0x7f004c0c4fa0 2024-01-20T08:58:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 -- 172.21.15.87:0/3772205408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c0c4fa0 msgr2=0x7f004c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 --2- 172.21.15.87:0/3772205408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c0c4fa0 0x7f004c159630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00400097a0 tx=0x7f004002b8d0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 -- 172.21.15.87:0/3772205408 shutdown_connections 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 --2- 172.21.15.87:0/3772205408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c0c4fa0 0x7f004c159630 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 --2- 172.21.15.87:0/3772205408 >> v2:172.21.15.116:3300/0 conn(0x7f004c153600 0x7f004c0c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 -- 172.21.15.87:0/3772205408 >> 172.21.15.87:0/3772205408 conn(0x7f004c08b5c0 msgr2=0x7f004c071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 -- 172.21.15.87:0/3772205408 shutdown_connections 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.280+0000 7f005274c640 1 -- 172.21.15.87:0/3772205408 wait complete. 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.281+0000 7f005274c640 1 Processor -- start 2024-01-20T08:58:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.281+0000 7f005274c640 1 -- start start 2024-01-20T08:58:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.282+0000 7f005274c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c072740 0x7f004c148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.282+0000 7f005274c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f004c0c4fa0 0x7f004c1ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.282+0000 7f005274c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f004c153600 0x7f004c1ef650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.282+0000 7f005274c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c159de0 con 0x7f004c153600 2024-01-20T08:58:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.282+0000 7f005274c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c1f2d40 con 0x7f004c0c4fa0 2024-01-20T08:58:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.282+0000 7f005274c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c1f3270 con 0x7f004c072740 2024-01-20T08:58:48.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.283+0000 7f0050ce9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f004c153600 0x7f004c1ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.283+0000 7f004b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f004c0c4fa0 0x7f004c1ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.283+0000 7f004bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c072740 0x7f004c148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.283+0000 7f004b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f004c0c4fa0 0x7f004c1ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49822/0 (socket says 172.21.15.87:49822) 2024-01-20T08:58:48.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.283+0000 7f004b7fe640 1 -- 172.21.15.87:0/780839963 learned_addr learned my addr 172.21.15.87:0/780839963 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.287+0000 7f004bfff640 1 -- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f004c0c4fa0 msgr2=0x7f004c1ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.287+0000 7f004bfff640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f004c0c4fa0 0x7f004c1ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.287+0000 7f004bfff640 1 -- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f004c153600 msgr2=0x7f004c1ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.287+0000 7f004bfff640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f004c153600 0x7f004c1ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.287+0000 7f004bfff640 1 -- 172.21.15.87:0/780839963 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040009400 con 0x7f004c072740 2024-01-20T08:58:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.287+0000 7f0050ce9640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f004c153600 0x7f004c1ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:58:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.289+0000 7f004bfff640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c072740 0x7f004c148810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e870 tx=0x7f003c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.292+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00ee60 con 0x7f004c072740 2024-01-20T08:58:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.292+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c00c940 con 0x7f004c072740 2024-01-20T08:58:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.293+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c153b60 con 0x7f004c072740 2024-01-20T08:58:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.293+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00ccd0 con 0x7f004c072740 2024-01-20T08:58:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.293+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c1f2f90 con 0x7f004c072740 2024-01-20T08:58:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.294+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c072560 con 0x7f004c072740 2024-01-20T08:58:48.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.542+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f003c013470 con 0x7f004c072740 2024-01-20T08:58:48.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.542+0000 7f0048ff9640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f003806aed0 0x7f003806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.543+0000 7f004b7fe640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f003806aed0 0x7f003806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.543+0000 7f004b7fe640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f003806aed0 0x7f003806d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f004c158290 tx=0x7f004003c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:48.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.545+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f003c00ce30 con 0x7f004c072740 2024-01-20T08:58:48.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.555+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f003c054d00 con 0x7f004c072740 2024-01-20T08:58:48.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.751+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f004c074290 con 0x7f004c072740 2024-01-20T08:58:48.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.756+0000 7f0048ff9640 1 -- 172.21.15.87:0/780839963 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f003c0210a0 con 0x7f004c072740 2024-01-20T08:58:48.757 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f003806aed0 msgr2=0x7f003806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f003806aed0 0x7f003806d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f004c158290 tx=0x7f004003c040 comp rx=0 tx=0).stop 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c072740 msgr2=0x7f004c148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c072740 0x7f004c148810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e870 tx=0x7f003c00ed30 comp rx=0 tx=0).stop 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 shutdown_connections 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f003806aed0 0x7f003806d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f004c153600 0x7f004c1ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f004c0c4fa0 0x7f004c1ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 --2- 172.21.15.87:0/780839963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f004c072740 0x7f004c148810 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 >> 172.21.15.87:0/780839963 conn(0x7f004c08b5c0 msgr2=0x7f004c06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 shutdown_connections 2024-01-20T08:58:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.758+0000 7f005274c640 1 -- 172.21.15.87:0/780839963 wait complete. 2024-01-20T08:58:48.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-20T08:58:48.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.872+0000 7f63dbad3640 1 Processor -- start 2024-01-20T08:58:48.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.872+0000 7f63dbad3640 1 -- start start 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63dbad3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d415b640 0x7f63d4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63dbad3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63d406efa0 0x7f63d40c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63dbad3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d4158330 con 0x7f63d4151690 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63dbad3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d406f840 con 0x7f63d406efa0 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63dbad3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4070050 con 0x7f63d415b640 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63d986f640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f63d4151690 0x7f63d415af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:48152/0 (socket says 172.21.15.87:48152) 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63d986f640 1 -- 172.21.15.87:0/1868670613 learned_addr learned my addr 172.21.15.87:0/1868670613 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:48.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.873+0000 7f63da070640 1 --2- 172.21.15.87:0/1868670613 >> v2:172.21.15.87:3300/0 conn(0x7f63d406efa0 0x7f63d40c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.875+0000 7f63d906e640 1 --2- 172.21.15.87:0/1868670613 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d415b640 0x7f63d4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.876+0000 7f63d886d640 1 -- 172.21.15.87:0/1868670613 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019100193 0 0) 0x7f63d0001110 con 0x7f63d4151690 2024-01-20T08:58:48.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.876+0000 7f63d886d640 1 -- 172.21.15.87:0/1868670613 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c0003520 con 0x7f63d4151690 2024-01-20T08:58:48.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.876+0000 7f63da070640 1 -- 172.21.15.87:0/1868670613 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d415b640 msgr2=0x7f63d4157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:48.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.876+0000 7f63da070640 1 --2- 172.21.15.87:0/1868670613 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d415b640 0x7f63d4157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.876+0000 7f63da070640 1 -- 172.21.15.87:0/1868670613 >> v1:172.21.15.116:6789/0 conn(0x7f63d4151690 legacy=0x7f63d415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.876+0000 7f63da070640 1 -- 172.21.15.87:0/1868670613 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d406e350 con 0x7f63d406efa0 2024-01-20T08:58:48.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.879+0000 7f63da070640 1 --2- 172.21.15.87:0/1868670613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d406efa0 0x7f63d40c8680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009d90 tx=0x7f63cc02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2dc15e2330b535 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:48.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.886+0000 7f63d886d640 1 -- 172.21.15.87:0/1868670613 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc02cce0 con 0x7f63d406efa0 2024-01-20T08:58:48.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63d886d640 1 -- 172.21.15.87:0/1868670613 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc02ce40 con 0x7f63d406efa0 2024-01-20T08:58:48.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63d886d640 1 -- 172.21.15.87:0/1868670613 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc034660 con 0x7f63d406efa0 2024-01-20T08:58:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 -- 172.21.15.87:0/1868670613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d406efa0 msgr2=0x7f63d40c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 --2- 172.21.15.87:0/1868670613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d406efa0 0x7f63d40c8680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009d90 tx=0x7f63cc02bf30 comp rx=0 tx=0).stop 2024-01-20T08:58:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 -- 172.21.15.87:0/1868670613 shutdown_connections 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 --2- 172.21.15.87:0/1868670613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d406efa0 0x7f63d40c8680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 --2- 172.21.15.87:0/1868670613 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d415b640 0x7f63d4157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 -- 172.21.15.87:0/1868670613 >> 172.21.15.87:0/1868670613 conn(0x7f63d4072390 msgr2=0x7f63d406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 -- 172.21.15.87:0/1868670613 shutdown_connections 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.887+0000 7f63dbad3640 1 -- 172.21.15.87:0/1868670613 wait complete. 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 Processor -- start 2024-01-20T08:58:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 -- start start 2024-01-20T08:58:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d406efa0 0x7f63d41f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 0x7f63d4159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d415b640 0x7f63d4159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4146590 con 0x7f63d4151690 2024-01-20T08:58:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4143860 con 0x7f63d415b640 2024-01-20T08:58:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.889+0000 7f63dbad3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d406ac50 con 0x7f63d406efa0 2024-01-20T08:58:48.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.890+0000 7f63d906e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 0x7f63d4159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.890+0000 7f63da070640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d415b640 0x7f63d4159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.890+0000 7f63d906e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 0x7f63d4159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59744/0 (socket says 172.21.15.87:59744) 2024-01-20T08:58:48.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.890+0000 7f63d986f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d406efa0 0x7f63d41f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:48.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.890+0000 7f63d906e640 1 -- 172.21.15.87:0/168842286 learned_addr learned my addr 172.21.15.87:0/168842286 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.894+0000 7f63d906e640 1 -- 172.21.15.87:0/168842286 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d406efa0 msgr2=0x7f63d41f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.894+0000 7f63d906e640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d406efa0 0x7f63d41f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.894+0000 7f63d906e640 1 -- 172.21.15.87:0/168842286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d415b640 msgr2=0x7f63d4159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.894+0000 7f63d906e640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d415b640 0x7f63d4159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.894+0000 7f63d906e640 1 -- 172.21.15.87:0/168842286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc009a70 con 0x7f63d4151690 2024-01-20T08:58:48.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.896+0000 7f63d906e640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 0x7f63d4159520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63c400d5c0 tx=0x7f63c400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.899+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40125f0 con 0x7f63d4151690 2024-01-20T08:58:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.899+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4012750 con 0x7f63d4151690 2024-01-20T08:58:48.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.899+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d40c5340 con 0x7f63d4151690 2024-01-20T08:58:48.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.899+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4012b00 con 0x7f63d4151690 2024-01-20T08:58:48.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.899+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d40c5900 con 0x7f63d4151690 2024-01-20T08:58:48.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:48.901+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d4074290 con 0x7f63d4151690 2024-01-20T08:58:49.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.141+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f63c4012c60 con 0x7f63d4151690 2024-01-20T08:58:49.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.141+0000 7f63ca7fc640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63b806af20 0x7f63b806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:49.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.141+0000 7f63d986f640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63b806af20 0x7f63b806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.142+0000 7f63d986f640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63b806af20 0x7f63b806d3e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005880 tx=0x7f63d0009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:49.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.163+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f63c40590c0 con 0x7f63d4151690 2024-01-20T08:58:49.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.163+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f63c4055400 con 0x7f63d4151690 2024-01-20T08:58:49.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.337+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f63d40c9830 con 0x7f63d4151690 2024-01-20T08:58:49.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.342+0000 7f63ca7fc640 1 -- 172.21.15.87:0/168842286 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f63c4058a20 con 0x7f63d4151690 2024-01-20T08:58:49.343 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:58:49.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63b806af20 msgr2=0x7f63b806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63b806af20 0x7f63b806d3e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005880 tx=0x7f63d0009040 comp rx=0 tx=0).stop 2024-01-20T08:58:49.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 msgr2=0x7f63d4159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 0x7f63d4159520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63c400d5c0 tx=0x7f63c400da80 comp rx=0 tx=0).stop 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 shutdown_connections 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f63b806af20 0x7f63b806d3e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d415b640 0x7f63d4159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63d4151690 0x7f63d4159520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 --2- 172.21.15.87:0/168842286 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d406efa0 0x7f63d41f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 >> 172.21.15.87:0/168842286 conn(0x7f63d4072390 msgr2=0x7f63d40c6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 shutdown_connections 2024-01-20T08:58:49.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.344+0000 7f63dbad3640 1 -- 172.21.15.87:0/168842286 wait complete. 2024-01-20T08:58:49.356 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-20T08:58:49.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.456+0000 7f41f7d5a640 1 Processor -- start 2024-01-20T08:58:49.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.456+0000 7f41f7d5a640 1 -- start start 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f7d5a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f015b640 0x7f41f0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f7d5a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f0158330 con 0x7f41f0151690 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f7d5a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f006f840 con 0x7f41f006efa0 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f7d5a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0070050 con 0x7f41f015b640 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f5af6640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f41f0151690 0x7f41f015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59054/0 (socket says 172.21.15.87:59054) 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f5af6640 1 -- 172.21.15.87:0/3111984821 learned_addr learned my addr 172.21.15.87:0/3111984821 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:49.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.457+0000 7f41f52f5640 1 --2- 172.21.15.87:0/3111984821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f015b640 0x7f41f0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:49.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946745020 0 0) 0x7f41ec001110 con 0x7f41f0151690 2024-01-20T08:58:49.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41dc003520 con 0x7f41f0151690 2024-01-20T08:58:49.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470344557 0 0) 0x7f41f006f840 con 0x7f41f006efa0 2024-01-20T08:58:49.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41ec001110 con 0x7f41f006efa0 2024-01-20T08:58:49.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f52f5640 1 -- 172.21.15.87:0/3111984821 >> v1:172.21.15.116:6789/0 conn(0x7f41f006efa0 legacy=0x7f41f00c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f52f5640 1 -- 172.21.15.87:0/3111984821 >> v1:172.21.15.87:6789/0 conn(0x7f41f0151690 legacy=0x7f41f015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.460+0000 7f41f52f5640 1 -- 172.21.15.87:0/3111984821 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f006e350 con 0x7f41f015b640 2024-01-20T08:58:49.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.462+0000 7f41f52f5640 1 --2- 172.21.15.87:0/3111984821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f015b640 0x7f41f0157df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f41e00097d0 tx=0x7f41e002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9515dfb849d0794d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:49.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.466+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0030880 con 0x7f41f015b640 2024-01-20T08:58:49.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.466+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e00309e0 con 0x7f41f015b640 2024-01-20T08:58:49.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.466+0000 7f41f4af4640 1 -- 172.21.15.87:0/3111984821 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0030d70 con 0x7f41f015b640 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 -- 172.21.15.87:0/3111984821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f015b640 msgr2=0x7f41f0157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/3111984821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f015b640 0x7f41f0157df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f41e00097d0 tx=0x7f41e002b9c0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 -- 172.21.15.87:0/3111984821 shutdown_connections 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/3111984821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f015b640 0x7f41f0157df0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 -- 172.21.15.87:0/3111984821 >> 172.21.15.87:0/3111984821 conn(0x7f41f0072390 msgr2=0x7f41f006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 -- 172.21.15.87:0/3111984821 shutdown_connections 2024-01-20T08:58:49.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.467+0000 7f41f7d5a640 1 -- 172.21.15.87:0/3111984821 wait complete. 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.468+0000 7f41f7d5a640 1 Processor -- start 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.468+0000 7f41f7d5a640 1 -- start start 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.469+0000 7f41f7d5a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41f006efa0 0x7f41f01f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.469+0000 7f41f7d5a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f0151690 0x7f41f00c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.469+0000 7f41f7d5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 0x7f41f00cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.469+0000 7f41f7d5a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00cc6b0 con 0x7f41f015b640 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.469+0000 7f41f7d5a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00cc900 con 0x7f41f006efa0 2024-01-20T08:58:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.469+0000 7f41f7d5a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0158a30 con 0x7f41f0151690 2024-01-20T08:58:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.470+0000 7f41f62f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 0x7f41f00cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.470+0000 7f41f52f5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f0151690 0x7f41f00c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.470+0000 7f41f5af6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41f006efa0 0x7f41f01f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.470+0000 7f41f62f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 0x7f41f00cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59752/0 (socket says 172.21.15.87:59752) 2024-01-20T08:58:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.470+0000 7f41f62f7640 1 -- 172.21.15.87:0/1726847174 learned_addr learned my addr 172.21.15.87:0/1726847174 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:49.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.473+0000 7f41f62f7640 1 -- 172.21.15.87:0/1726847174 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f0151690 msgr2=0x7f41f00c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.473+0000 7f41f62f7640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f0151690 0x7f41f00c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.473+0000 7f41f62f7640 1 -- 172.21.15.87:0/1726847174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41f006efa0 msgr2=0x7f41f01f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.473+0000 7f41f62f7640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41f006efa0 0x7f41f01f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.473+0000 7f41f62f7640 1 -- 172.21.15.87:0/1726847174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0009400 con 0x7f41f015b640 2024-01-20T08:58:49.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.476+0000 7f41f62f7640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 0x7f41f00cb690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c790 tx=0x7f41e800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:49.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.479+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800cea0 con 0x7f41f015b640 2024-01-20T08:58:49.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.479+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8003b70 con 0x7f41f015b640 2024-01-20T08:58:49.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.479+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800a850 con 0x7f41f015b640 2024-01-20T08:58:49.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.480+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f0158d10 con 0x7f41f015b640 2024-01-20T08:58:49.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.480+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f00cbbd0 con 0x7f41f015b640 2024-01-20T08:58:49.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.481+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f00c2f00 con 0x7f41f015b640 2024-01-20T08:58:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.499+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f41e800a9b0 con 0x7f41f015b640 2024-01-20T08:58:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.499+0000 7f41e67fc640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f41d406af20 0x7f41d406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.499+0000 7f41f5af6640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f41d406af20 0x7f41d406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.500+0000 7f41f5af6640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f41d406af20 0x7f41d406d3e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000f60 tx=0x7f41ec007cf0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.503+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f41e8058b20 con 0x7f41f015b640 2024-01-20T08:58:49.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.740+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f41e8054e60 con 0x7f41f015b640 2024-01-20T08:58:49.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.934+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41f00c26e0 con 0x7f41f015b640 2024-01-20T08:58:49.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.940+0000 7f41e67fc640 1 -- 172.21.15.87:0/1726847174 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f41f00c26e0 con 0x7f41f015b640 2024-01-20T08:58:49.941 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-01-20T08:58:49.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f41d406af20 msgr2=0x7f41d406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f41d406af20 0x7f41d406d3e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000f60 tx=0x7f41ec007cf0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 msgr2=0x7f41f00cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:49.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 0x7f41f00cb690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c790 tx=0x7f41e800cc50 comp rx=0 tx=0).stop 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 shutdown_connections 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f41d406af20 0x7f41d406d3e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f015b640 0x7f41f00cb690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41f0151690 0x7f41f00c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 --2- 172.21.15.87:0/1726847174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41f006efa0 0x7f41f01f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 >> 172.21.15.87:0/1726847174 conn(0x7f41f0072390 msgr2=0x7f41f006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 shutdown_connections 2024-01-20T08:58:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:49.942+0000 7f41f7d5a640 1 -- 172.21.15.87:0/1726847174 wait complete. 2024-01-20T08:58:49.955 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-01-20T08:58:49.956 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-20T08:58:49.971 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:58:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.109+0000 7f917f3d7640 1 Processor -- start 2024-01-20T08:58:50.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.109+0000 7f917f3d7640 1 -- start start 2024-01-20T08:58:50.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.110+0000 7f917f3d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc090 0x7f917806af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.110+0000 7f917f3d7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9178159770 0x7f91780c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.110+0000 7f917f3d7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91780cc470 0x7f917815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.110+0000 7f917f3d7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917815d2c0 con 0x7f91780cc470 2024-01-20T08:58:50.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.110+0000 7f917f3d7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91780c6e10 con 0x7f9178159770 2024-01-20T08:58:50.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.110+0000 7f917f3d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917806b540 con 0x7f91780cc090 2024-01-20T08:58:50.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.111+0000 7f917c972640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9178159770 0x7f91780c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.111+0000 7f917d974640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91780cc470 0x7f917815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.111+0000 7f917d173640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc090 0x7f917806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.111+0000 7f917c972640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9178159770 0x7f91780c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49848/0 (socket says 172.21.15.87:49848) 2024-01-20T08:58:50.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.111+0000 7f917c972640 1 -- 172.21.15.87:0/2922086731 learned_addr learned my addr 172.21.15.87:0/2922086731 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.113+0000 7f917c972640 1 -- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc090 msgr2=0x7f917806af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.113+0000 7f917c972640 1 --2- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc090 0x7f917806af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.113+0000 7f917c972640 1 -- 172.21.15.87:0/2922086731 >> v2:172.21.15.87:3300/0 conn(0x7f91780cc470 msgr2=0x7f917815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.113+0000 7f917c972640 1 --2- 172.21.15.87:0/2922086731 >> v2:172.21.15.87:3300/0 conn(0x7f91780cc470 0x7f917815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.113+0000 7f917c972640 1 -- 172.21.15.87:0/2922086731 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917806fac0 con 0x7f9178159770 2024-01-20T08:58:50.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.115+0000 7f917c972640 1 --2- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9178159770 0x7f91780c68d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9178071950 tx=0x7f916802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3919d33ab88efded server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:50.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.119+0000 7f916ffff640 1 -- 172.21.15.87:0/2922086731 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168030860 con 0x7f9178159770 2024-01-20T08:58:50.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.119+0000 7f916ffff640 1 -- 172.21.15.87:0/2922086731 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91680309c0 con 0x7f9178159770 2024-01-20T08:58:50.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.119+0000 7f916ffff640 1 -- 172.21.15.87:0/2922086731 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168030d50 con 0x7f9178159770 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 -- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9178159770 msgr2=0x7f91780c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 --2- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9178159770 0x7f91780c68d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9178071950 tx=0x7f916802b9a0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 -- 172.21.15.87:0/2922086731 shutdown_connections 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 --2- 172.21.15.87:0/2922086731 >> v2:172.21.15.87:3300/0 conn(0x7f91780cc470 0x7f917815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 --2- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9178159770 0x7f91780c68d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 --2- 172.21.15.87:0/2922086731 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc090 0x7f917806af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 -- 172.21.15.87:0/2922086731 >> 172.21.15.87:0/2922086731 conn(0x7f9178071ff0 msgr2=0x7f9178072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 -- 172.21.15.87:0/2922086731 shutdown_connections 2024-01-20T08:58:50.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.120+0000 7f917f3d7640 1 -- 172.21.15.87:0/2922086731 wait complete. 2024-01-20T08:58:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.121+0000 7f917f3d7640 1 Processor -- start 2024-01-20T08:58:50.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.121+0000 7f917f3d7640 1 -- start start 2024-01-20T08:58:50.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917f3d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91780cc090 0x7f917815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917f3d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc470 0x7f91780c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917f3d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 0x7f917814dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917f3d7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178154420 con 0x7f9178159770 2024-01-20T08:58:50.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917f3d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178158180 con 0x7f91780cc090 2024-01-20T08:58:50.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917f3d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91780c73d0 con 0x7f91780cc470 2024-01-20T08:58:50.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917d974640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 0x7f917814dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917d974640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 0x7f917814dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59780/0 (socket says 172.21.15.87:59780) 2024-01-20T08:58:50.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.122+0000 7f917d974640 1 -- 172.21.15.87:0/3419374697 learned_addr learned my addr 172.21.15.87:0/3419374697 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:50.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.123+0000 7f917d173640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91780cc090 0x7f917815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.123+0000 7f917c972640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc470 0x7f91780c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.126+0000 7f917d974640 1 -- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc470 msgr2=0x7f91780c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.126+0000 7f917d974640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc470 0x7f91780c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.126+0000 7f917d974640 1 -- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91780cc090 msgr2=0x7f917815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.126+0000 7f917d974640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91780cc090 0x7f917815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.126+0000 7f917d974640 1 -- 172.21.15.87:0/3419374697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680094d0 con 0x7f9178159770 2024-01-20T08:58:50.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.128+0000 7f917d974640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 0x7f917814dd00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f917400e7c0 tx=0x7f917400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:50.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.131+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740094b0 con 0x7f9178159770 2024-01-20T08:58:50.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.131+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400c730 con 0x7f9178159770 2024-01-20T08:58:50.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.131+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400cae0 con 0x7f9178159770 2024-01-20T08:58:50.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.131+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91781552a0 con 0x7f9178159770 2024-01-20T08:58:50.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.132+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91780c9290 con 0x7f9178159770 2024-01-20T08:58:50.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.133+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9178074290 con 0x7f9178159770 2024-01-20T08:58:50.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.152+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f917400cc40 con 0x7f9178159770 2024-01-20T08:58:50.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.152+0000 7f916dffb640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f915c06af20 0x7f915c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.152+0000 7f917d173640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f915c06af20 0x7f915c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.153+0000 7f917d173640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f915c06af20 0x7f915c06d3e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9170011200 tx=0x7f917000d040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:50.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.156+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9174058980 con 0x7f9178159770 2024-01-20T08:58:50.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.391+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9174054cc0 con 0x7f9178159770 2024-01-20T08:58:50.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.563+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91780c1120 con 0x7f9178159770 2024-01-20T08:58:50.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.601+0000 7f916dffb640 1 -- 172.21.15.87:0/3419374697 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7f9174021090 con 0x7f9178159770 2024-01-20T08:58:50.604 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:58:50.604 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:58:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f915c06af20 msgr2=0x7f915c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f915c06af20 0x7f915c06d3e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9170011200 tx=0x7f917000d040 comp rx=0 tx=0).stop 2024-01-20T08:58:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 msgr2=0x7f917814dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 0x7f917814dd00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f917400e7c0 tx=0x7f917400ec80 comp rx=0 tx=0).stop 2024-01-20T08:58:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 shutdown_connections 2024-01-20T08:58:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f915c06af20 0x7f915c06d3e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9178159770 0x7f917814dd00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f91780cc470 0x7f91780c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 --2- 172.21.15.87:0/3419374697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91780cc090 0x7f917815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 >> 172.21.15.87:0/3419374697 conn(0x7f9178071ff0 msgr2=0x7f91780c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 shutdown_connections 2024-01-20T08:58:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.604+0000 7f917f3d7640 1 -- 172.21.15.87:0/3419374697 wait complete. 2024-01-20T08:58:50.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-20T08:30:44.696145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-20T08:31:06.500533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-20T08:31:08.175669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-20T08:31:13.903374+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-20T08:58:50.618 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-20T08:58:50.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.718+0000 7f3d6bfd7640 1 Processor -- start 2024-01-20T08:58:50.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.718+0000 7f3d6bfd7640 1 -- start start 2024-01-20T08:58:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.719+0000 7f3d6bfd7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d6414cc40 0x7f3d64156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.719+0000 7f3d6bfd7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d6415ec50 0x7f3d64161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.719+0000 7f3d6bfd7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d64156720 con 0x7f3d640bfe70 2024-01-20T08:58:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.719+0000 7f3d6bfd7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d64072b30 con 0x7f3d6414cc40 2024-01-20T08:58:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.719+0000 7f3d6bfd7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c19c0 con 0x7f3d6415ec50 2024-01-20T08:58:50.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.720+0000 7f3d69572640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3d640bfe70 0x7f3d640c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59066/0 (socket says 172.21.15.87:59066) 2024-01-20T08:58:50.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.720+0000 7f3d69572640 1 -- 172.21.15.87:0/2505570056 learned_addr learned my addr 172.21.15.87:0/2505570056 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:50.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.720+0000 7f3d69d73640 1 --2- 172.21.15.87:0/2505570056 >> v2:172.21.15.116:3300/0 conn(0x7f3d6414cc40 0x7f3d64156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.720+0000 7f3d6a574640 1 --2- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d6415ec50 0x7f3d64161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.722+0000 7f3d68d71640 1 -- 172.21.15.87:0/2505570056 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309432308 0 0) 0x7f3d64156720 con 0x7f3d640bfe70 2024-01-20T08:58:50.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.722+0000 7f3d68d71640 1 -- 172.21.15.87:0/2505570056 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4c003520 con 0x7f3d640bfe70 2024-01-20T08:58:50.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.722+0000 7f3d69d73640 1 -- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d6415ec50 msgr2=0x7f3d64161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:50.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.722+0000 7f3d69d73640 1 --2- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d6415ec50 0x7f3d64161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.722+0000 7f3d69d73640 1 -- 172.21.15.87:0/2505570056 >> v1:172.21.15.87:6789/0 conn(0x7f3d640bfe70 legacy=0x7f3d640c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.722+0000 7f3d69d73640 1 -- 172.21.15.87:0/2505570056 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d641563f0 con 0x7f3d6414cc40 2024-01-20T08:58:50.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.724+0000 7f3d69d73640 1 --2- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d64156bb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009d90 tx=0x7f3d5c02bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4178ae02adeeddfa server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:50.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.728+0000 7f3d68d71640 1 -- 172.21.15.87:0/2505570056 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c02cce0 con 0x7f3d6414cc40 2024-01-20T08:58:50.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.728+0000 7f3d68d71640 1 -- 172.21.15.87:0/2505570056 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c02ce40 con 0x7f3d6414cc40 2024-01-20T08:58:50.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.728+0000 7f3d68d71640 1 -- 172.21.15.87:0/2505570056 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c034660 con 0x7f3d6414cc40 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 msgr2=0x7f3d64156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d64156bb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009d90 tx=0x7f3d5c02bf30 comp rx=0 tx=0).stop 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2505570056 shutdown_connections 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d6415ec50 0x7f3d64161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2505570056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d64156bb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2505570056 >> 172.21.15.87:0/2505570056 conn(0x7f3d6406a9a0 msgr2=0x7f3d6406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2505570056 shutdown_connections 2024-01-20T08:58:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.729+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2505570056 wait complete. 2024-01-20T08:58:50.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.730+0000 7f3d6bfd7640 1 Processor -- start 2024-01-20T08:58:50.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.730+0000 7f3d6bfd7640 1 -- start start 2024-01-20T08:58:50.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6bfd7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d640bfe70 0x7f3d6414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6bfd7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d640c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6bfd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 0x7f3d640c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6bfd7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c2310 con 0x7f3d6415ec50 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6bfd7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6414f1c0 con 0x7f3d6414cc40 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6bfd7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6414e640 con 0x7f3d640bfe70 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d6a574640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 0x7f3d640c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.731+0000 7f3d69d73640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d640bfe70 0x7f3d6414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.732+0000 7f3d69572640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d640c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.732+0000 7f3d6a574640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 0x7f3d640c27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59786/0 (socket says 172.21.15.87:59786) 2024-01-20T08:58:50.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.732+0000 7f3d6a574640 1 -- 172.21.15.87:0/2912894698 learned_addr learned my addr 172.21.15.87:0/2912894698 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:50.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.735+0000 7f3d6a574640 1 -- 172.21.15.87:0/2912894698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d640bfe70 msgr2=0x7f3d6414d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.735+0000 7f3d6a574640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d640bfe70 0x7f3d6414d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.735+0000 7f3d6a574640 1 -- 172.21.15.87:0/2912894698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 msgr2=0x7f3d640c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:50.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.735+0000 7f3d6a574640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d640c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:50.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.735+0000 7f3d6a574640 1 -- 172.21.15.87:0/2912894698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c009a70 con 0x7f3d6415ec50 2024-01-20T08:58:50.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.737+0000 7f3d6a574640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 0x7f3d640c27a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400e7f0 tx=0x7f3d5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:50.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.740+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d540094b0 con 0x7f3d6415ec50 2024-01-20T08:58:50.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.740+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5400c730 con 0x7f3d6415ec50 2024-01-20T08:58:50.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.740+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6414e920 con 0x7f3d6415ec50 2024-01-20T08:58:50.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.740+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400cae0 con 0x7f3d6415ec50 2024-01-20T08:58:50.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.740+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6414eee0 con 0x7f3d6415ec50 2024-01-20T08:58:50.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.741+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640c24c0 con 0x7f3d6415ec50 2024-01-20T08:58:50.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.760+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3d5400cc40 con 0x7f3d6415ec50 2024-01-20T08:58:50.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.761+0000 7f3d627fc640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d5006af50 0x7f3d5006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:50.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.761+0000 7f3d69d73640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d5006af50 0x7f3d5006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:50.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.761+0000 7f3d69d73640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d5006af50 0x7f3d5006d410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c036000 tx=0x7f3d5c030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:50.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.764+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f3d54058850 con 0x7f3d6415ec50 2024-01-20T08:58:50.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:50.996+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3d54054b90 con 0x7f3d6415ec50 2024-01-20T08:58:51.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.176+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d640c24c0 con 0x7f3d6415ec50 2024-01-20T08:58:51.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.181+0000 7f3d627fc640 1 -- 172.21.15.87:0/2912894698 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d640c24c0 con 0x7f3d6415ec50 2024-01-20T08:58:51.183 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-01-20T08:58:51.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d5006af50 msgr2=0x7f3d5006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d5006af50 0x7f3d5006d410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c036000 tx=0x7f3d5c030000 comp rx=0 tx=0).stop 2024-01-20T08:58:51.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 msgr2=0x7f3d640c27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 0x7f3d640c27a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400e7f0 tx=0x7f3d5400ecb0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 shutdown_connections 2024-01-20T08:58:51.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f3d5006af50 0x7f3d5006d410 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d6415ec50 0x7f3d640c27a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d6414cc40 0x7f3d640c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 --2- 172.21.15.87:0/2912894698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3d640bfe70 0x7f3d6414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 >> 172.21.15.87:0/2912894698 conn(0x7f3d6406a9a0 msgr2=0x7f3d640c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:51.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 shutdown_connections 2024-01-20T08:58:51.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.183+0000 7f3d6bfd7640 1 -- 172.21.15.87:0/2912894698 wait complete. 2024-01-20T08:58:51.197 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-20T08:58:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.297+0000 7f4a47f61640 1 Processor -- start 2024-01-20T08:58:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.298+0000 7f4a47f61640 1 -- start start 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.298+0000 7f4a47f61640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a400b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.298+0000 7f4a47f61640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a400ca070 con 0x7f4a400c9520 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.298+0000 7f4a47f61640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a40157ef0 con 0x7f4a400c90e0 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.298+0000 7f4a47f61640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a400b0520 con 0x7f4a401517f0 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.299+0000 7f4a454fc640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4a400c90e0 0x7f4a40157460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:48172/0 (socket says 172.21.15.87:48172) 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.299+0000 7f4a454fc640 1 -- 172.21.15.87:0/1805930484 learned_addr learned my addr 172.21.15.87:0/1805930484 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.299+0000 7f4a45cfd640 1 --2- 172.21.15.87:0/1805930484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a400b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.301+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268396848 0 0) 0x7f4a40157ef0 con 0x7f4a400c90e0 2024-01-20T08:58:51.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.301+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a2c003520 con 0x7f4a400c90e0 2024-01-20T08:58:51.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.301+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084314364 0 0) 0x7f4a400ca070 con 0x7f4a400c9520 2024-01-20T08:58:51.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.302+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a40157ef0 con 0x7f4a400c9520 2024-01-20T08:58:51.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.302+0000 7f4a45cfd640 1 -- 172.21.15.87:0/1805930484 >> v1:172.21.15.116:6789/0 conn(0x7f4a400c90e0 legacy=0x7f4a40157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.302+0000 7f4a45cfd640 1 -- 172.21.15.87:0/1805930484 >> v1:172.21.15.87:6789/0 conn(0x7f4a400c9520 legacy=0x7f4a400ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.302+0000 7f4a45cfd640 1 -- 172.21.15.87:0/1805930484 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a401581d0 con 0x7f4a401517f0 2024-01-20T08:58:51.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.304+0000 7f4a45cfd640 1 --2- 172.21.15.87:0/1805930484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a400b0dd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009dc0 tx=0x7f4a3802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9e8aca70dc8dba server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.307+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3802ccd0 con 0x7f4a401517f0 2024-01-20T08:58:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.307+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3802ce30 con 0x7f4a401517f0 2024-01-20T08:58:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.307+0000 7f4a44cfb640 1 -- 172.21.15.87:0/1805930484 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38034660 con 0x7f4a401517f0 2024-01-20T08:58:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 -- 172.21.15.87:0/1805930484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 msgr2=0x7f4a400b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 --2- 172.21.15.87:0/1805930484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a400b0dd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009dc0 tx=0x7f4a3802bf20 comp rx=0 tx=0).stop 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 -- 172.21.15.87:0/1805930484 shutdown_connections 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 --2- 172.21.15.87:0/1805930484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a400b0dd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 -- 172.21.15.87:0/1805930484 >> 172.21.15.87:0/1805930484 conn(0x7f4a4006a5b0 msgr2=0x7f4a4006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 -- 172.21.15.87:0/1805930484 shutdown_connections 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.308+0000 7f4a47f61640 1 -- 172.21.15.87:0/1805930484 wait complete. 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.309+0000 7f4a47f61640 1 Processor -- start 2024-01-20T08:58:51.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 -- start start 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a400c90e0 0x7f4a40152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a400c9520 0x7f4a40070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a401ef450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40152780 con 0x7f4a400c90e0 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40158620 con 0x7f4a400c9520 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.310+0000 7f4a47f61640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401588c0 con 0x7f4a401517f0 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.311+0000 7f4a45cfd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a400c90e0 0x7f4a40152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.311+0000 7f4a454fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a400c9520 0x7f4a40070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.311+0000 7f4a464fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a401ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.311+0000 7f4a454fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a400c9520 0x7f4a40070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49880/0 (socket says 172.21.15.87:49880) 2024-01-20T08:58:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.311+0000 7f4a454fc640 1 -- 172.21.15.87:0/3583289245 learned_addr learned my addr 172.21.15.87:0/3583289245 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.314+0000 7f4a45cfd640 1 -- 172.21.15.87:0/3583289245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 msgr2=0x7f4a401ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.314+0000 7f4a45cfd640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a401ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.314+0000 7f4a45cfd640 1 -- 172.21.15.87:0/3583289245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a400c9520 msgr2=0x7f4a40070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.314+0000 7f4a45cfd640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a400c9520 0x7f4a40070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.314+0000 7f4a45cfd640 1 -- 172.21.15.87:0/3583289245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a38009a70 con 0x7f4a400c90e0 2024-01-20T08:58:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.316+0000 7f4a45cfd640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a400c90e0 0x7f4a40152940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4a38033000 tx=0x7f4a38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.320+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3803e610 con 0x7f4a400c90e0 2024-01-20T08:58:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.320+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3803e790 con 0x7f4a400c90e0 2024-01-20T08:58:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.320+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3803eb40 con 0x7f4a400c90e0 2024-01-20T08:58:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.320+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4014ed90 con 0x7f4a400c90e0 2024-01-20T08:58:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.320+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a401f3120 con 0x7f4a400c90e0 2024-01-20T08:58:51.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.321+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a401f3310 con 0x7f4a400c90e0 2024-01-20T08:58:51.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.560+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4a3803b050 con 0x7f4a400c90e0 2024-01-20T08:58:51.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.560+0000 7f4a367fc640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4a2406af20 0x7f4a2406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.560+0000 7f4a454fc640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4a2406af20 0x7f4a2406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.561+0000 7f4a454fc640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4a2406af20 0x7f4a2406d3e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4a4014cd60 tx=0x7f4a30009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:51.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.574+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4a3807ae10 con 0x7f4a400c90e0 2024-01-20T08:58:51.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.574+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4a38077150 con 0x7f4a400c90e0 2024-01-20T08:58:51.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.744+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a40074290 con 0x7f4a400c90e0 2024-01-20T08:58:51.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.750+0000 7f4a367fc640 1 -- 172.21.15.87:0/3583289245 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a3804d0a0 con 0x7f4a400c90e0 2024-01-20T08:58:51.753 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:58:51.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4a2406af20 msgr2=0x7f4a2406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4a2406af20 0x7f4a2406d3e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4a4014cd60 tx=0x7f4a30009040 comp rx=0 tx=0).stop 2024-01-20T08:58:51.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a400c90e0 msgr2=0x7f4a40152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a400c90e0 0x7f4a40152940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4a38033000 tx=0x7f4a38004a40 comp rx=0 tx=0).stop 2024-01-20T08:58:51.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 shutdown_connections 2024-01-20T08:58:51.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4a2406af20 0x7f4a2406d3e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a401517f0 0x7f4a401ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a400c9520 0x7f4a40070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 --2- 172.21.15.87:0/3583289245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a400c90e0 0x7f4a40152940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 >> 172.21.15.87:0/3583289245 conn(0x7f4a4006a5b0 msgr2=0x7f4a40157670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:51.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 shutdown_connections 2024-01-20T08:58:51.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.753+0000 7f4a47f61640 1 -- 172.21.15.87:0/3583289245 wait complete. 2024-01-20T08:58:51.766 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-20T08:58:51.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.866+0000 7ff007705640 1 Processor -- start 2024-01-20T08:58:51.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.866+0000 7ff007705640 1 -- start start 2024-01-20T08:58:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.867+0000 7ff007705640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 0x7ff0000712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.867+0000 7ff007705640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff0000adc10 0x7ff0000c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.867+0000 7ff007705640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00006f590 con 0x7ff0000c5b10 2024-01-20T08:58:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.867+0000 7ff007705640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0000b06d0 con 0x7ff0000adc10 2024-01-20T08:58:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.867+0000 7ff007705640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00015aea0 con 0x7ff0001517d0 2024-01-20T08:58:51.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.868+0000 7ff0054a1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 0x7ff0000712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.868+0000 7ff005ca2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff0000adc10 0x7ff0000c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.868+0000 7ff0054a1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 0x7ff0000712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:34708/0 (socket says 172.21.15.87:34708) 2024-01-20T08:58:51.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.868+0000 7ff0054a1640 1 -- 172.21.15.87:0/2559137090 learned_addr learned my addr 172.21.15.87:0/2559137090 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:51.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.871+0000 7ff0054a1640 1 -- 172.21.15.87:0/2559137090 >> v2:172.21.15.116:3300/0 conn(0x7ff0000adc10 msgr2=0x7ff0000c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:51.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.871+0000 7ff0054a1640 1 --2- 172.21.15.87:0/2559137090 >> v2:172.21.15.116:3300/0 conn(0x7ff0000adc10 0x7ff0000c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.871+0000 7ff0054a1640 1 -- 172.21.15.87:0/2559137090 >> v1:172.21.15.116:6789/0 conn(0x7ff0000c5b10 legacy=0x7ff00015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.871+0000 7ff0054a1640 1 -- 172.21.15.87:0/2559137090 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000071800 con 0x7ff0001517d0 2024-01-20T08:58:51.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.873+0000 7ff0054a1640 1 --2- 172.21.15.87:0/2559137090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 0x7ff0000712c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feffc009e50 tx=0x7feffc02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c4976e5a54180e server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:51.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.877+0000 7feff7fff640 1 -- 172.21.15.87:0/2559137090 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc02c9e0 con 0x7ff0001517d0 2024-01-20T08:58:51.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.877+0000 7feff7fff640 1 -- 172.21.15.87:0/2559137090 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc02cb40 con 0x7ff0001517d0 2024-01-20T08:58:51.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.877+0000 7feff7fff640 1 -- 172.21.15.87:0/2559137090 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc0343f0 con 0x7ff0001517d0 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.877+0000 7ff007705640 1 -- 172.21.15.87:0/2559137090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 msgr2=0x7ff0000712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.877+0000 7ff007705640 1 --2- 172.21.15.87:0/2559137090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 0x7ff0000712c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feffc009e50 tx=0x7feffc02be40 comp rx=0 tx=0).stop 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.878+0000 7ff007705640 1 -- 172.21.15.87:0/2559137090 shutdown_connections 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.878+0000 7ff007705640 1 --2- 172.21.15.87:0/2559137090 >> v2:172.21.15.116:3300/0 conn(0x7ff0000adc10 0x7ff0000c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.878+0000 7ff007705640 1 --2- 172.21.15.87:0/2559137090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0001517d0 0x7ff0000712c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.878+0000 7ff007705640 1 -- 172.21.15.87:0/2559137090 >> 172.21.15.87:0/2559137090 conn(0x7ff00006abe0 msgr2=0x7ff00006ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.878+0000 7ff007705640 1 -- 172.21.15.87:0/2559137090 shutdown_connections 2024-01-20T08:58:51.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.878+0000 7ff007705640 1 -- 172.21.15.87:0/2559137090 wait complete. 2024-01-20T08:58:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 Processor -- start 2024-01-20T08:58:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 -- start start 2024-01-20T08:58:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0000adc10 0x7ff000143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0000c5b10 0x7ff0000c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 0x7ff0000c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00014f450 con 0x7ff0000c5b10 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001436a0 con 0x7ff0001517d0 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.879+0000 7ff007705640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00014dfb0 con 0x7ff0000adc10 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.880+0000 7ff005ca2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 0x7ff0000c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.880+0000 7ff0054a1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0000adc10 0x7ff000143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.880+0000 7ff004ca0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0000c5b10 0x7ff0000c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.880+0000 7ff005ca2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 0x7ff0000c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49886/0 (socket says 172.21.15.87:49886) 2024-01-20T08:58:51.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.880+0000 7ff005ca2640 1 -- 172.21.15.87:0/303809868 learned_addr learned my addr 172.21.15.87:0/303809868 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:51.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.884+0000 7ff005ca2640 1 -- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0000adc10 msgr2=0x7ff000143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.884+0000 7ff005ca2640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0000adc10 0x7ff000143160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.884+0000 7ff005ca2640 1 -- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0000c5b10 msgr2=0x7ff0000c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:51.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.884+0000 7ff005ca2640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0000c5b10 0x7ff0000c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:51.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.884+0000 7ff005ca2640 1 -- 172.21.15.87:0/303809868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc009b30 con 0x7ff0001517d0 2024-01-20T08:58:51.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.886+0000 7ff005ca2640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 0x7ff0000c4be0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feff800e960 tx=0x7feff800ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:51.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.890+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8009690 con 0x7ff0001517d0 2024-01-20T08:58:51.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.890+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff800c980 con 0x7ff0001517d0 2024-01-20T08:58:51.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.890+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00014eab0 con 0x7ff0001517d0 2024-01-20T08:58:51.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.890+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800cd30 con 0x7ff0001517d0 2024-01-20T08:58:51.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.890+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00014f070 con 0x7ff0001517d0 2024-01-20T08:58:51.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.891+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00014f590 con 0x7ff0001517d0 2024-01-20T08:58:51.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.911+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7feff8013470 con 0x7ff0001517d0 2024-01-20T08:58:51.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.911+0000 7feff5ffb640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fefe406af50 0x7fefe406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:51.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.911+0000 7ff0054a1640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fefe406af50 0x7fefe406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:51.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:51.911+0000 7ff0054a1640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fefe406af50 0x7fefe406d410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feffc035000 tx=0x7feffc030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.139+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7feff8058a50 con 0x7ff0001517d0 2024-01-20T08:58:52.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.150+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7feff8054d90 con 0x7ff0001517d0 2024-01-20T08:58:52.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.347+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff00014f590 con 0x7ff0001517d0 2024-01-20T08:58:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.352+0000 7feff5ffb640 1 -- 172.21.15.87:0/303809868 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff00014f590 con 0x7ff0001517d0 2024-01-20T08:58:52.354 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-01-20T08:58:52.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fefe406af50 msgr2=0x7fefe406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fefe406af50 0x7fefe406d410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feffc035000 tx=0x7feffc030000 comp rx=0 tx=0).stop 2024-01-20T08:58:52.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 msgr2=0x7ff0000c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 0x7ff0000c4be0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feff800e960 tx=0x7feff800ee20 comp rx=0 tx=0).stop 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 shutdown_connections 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fefe406af50 0x7fefe406d410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0001517d0 0x7ff0000c4be0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0000c5b10 0x7ff0000c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 --2- 172.21.15.87:0/303809868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0000adc10 0x7ff000143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 >> 172.21.15.87:0/303809868 conn(0x7ff00006abe0 msgr2=0x7ff000158790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 shutdown_connections 2024-01-20T08:58:52.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.354+0000 7ff007705640 1 -- 172.21.15.87:0/303809868 wait complete. 2024-01-20T08:58:52.368 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-20T08:58:52.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.468+0000 7f267799b640 1 Processor -- start 2024-01-20T08:58:52.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.469+0000 7f267799b640 1 -- start start 2024-01-20T08:58:52.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.469+0000 7f267799b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f267006ef60 0x7f267006f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:52.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.469+0000 7f267799b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2670159710 0x7f2670159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:52.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.469+0000 7f267799b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267015cb70 con 0x7f2670153800 2024-01-20T08:58:52.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.469+0000 7f267799b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26700c2730 con 0x7f2670159710 2024-01-20T08:58:52.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.469+0000 7f267799b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26700c9430 con 0x7f267006ef60 2024-01-20T08:58:52.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.470+0000 7f2674f36640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f267006ef60 0x7f267006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:52.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.471+0000 7f2675f38640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2670159710 0x7f2670159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:52.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.471+0000 7f2675737640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2670153800 0x7f26700c99d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:48186/0 (socket says 172.21.15.87:48186) 2024-01-20T08:58:52.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.471+0000 7f2675737640 1 -- 172.21.15.87:0/3066266357 learned_addr learned my addr 172.21.15.87:0/3066266357 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:58:52.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.473+0000 7f2674f36640 1 -- 172.21.15.87:0/3066266357 >> v2:172.21.15.116:3300/0 conn(0x7f2670159710 msgr2=0x7f2670159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:52.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.473+0000 7f2674f36640 1 --2- 172.21.15.87:0/3066266357 >> v2:172.21.15.116:3300/0 conn(0x7f2670159710 0x7f2670159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.473+0000 7f2674f36640 1 -- 172.21.15.87:0/3066266357 >> v1:172.21.15.116:6789/0 conn(0x7f2670153800 legacy=0x7f26700c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.473+0000 7f2674f36640 1 -- 172.21.15.87:0/3066266357 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267015d9d0 con 0x7f267006ef60 2024-01-20T08:58:52.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.475+0000 7f2674f36640 1 --2- 172.21.15.87:0/3066266357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f267006ef60 0x7f267006f380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f266c0097d0 tx=0x7f266c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb7dbb982982e7a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:52.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.478+0000 7f2667fff640 1 -- 172.21.15.87:0/3066266357 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c031800 con 0x7f267006ef60 2024-01-20T08:58:52.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f2667fff640 1 -- 172.21.15.87:0/3066266357 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c031960 con 0x7f267006ef60 2024-01-20T08:58:52.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f2667fff640 1 -- 172.21.15.87:0/3066266357 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c031cf0 con 0x7f267006ef60 2024-01-20T08:58:52.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 -- 172.21.15.87:0/3066266357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f267006ef60 msgr2=0x7f267006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 --2- 172.21.15.87:0/3066266357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f267006ef60 0x7f267006f380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f266c0097d0 tx=0x7f266c02b8a0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 -- 172.21.15.87:0/3066266357 shutdown_connections 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 --2- 172.21.15.87:0/3066266357 >> v2:172.21.15.116:3300/0 conn(0x7f2670159710 0x7f2670159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 --2- 172.21.15.87:0/3066266357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f267006ef60 0x7f267006f380 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 -- 172.21.15.87:0/3066266357 >> 172.21.15.87:0/3066266357 conn(0x7f2670070de0 msgr2=0x7f26700711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 -- 172.21.15.87:0/3066266357 shutdown_connections 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.480+0000 7f267799b640 1 -- 172.21.15.87:0/3066266357 wait complete. 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.481+0000 7f267799b640 1 Processor -- start 2024-01-20T08:58:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.481+0000 7f267799b640 1 -- start start 2024-01-20T08:58:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.482+0000 7f267799b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 0x7f267015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.482+0000 7f267799b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2670153800 0x7f26701e6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.482+0000 7f267799b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2670159710 0x7f2670147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.482+0000 7f267799b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2670158ab0 con 0x7f2670153800 2024-01-20T08:58:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.482+0000 7f267799b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267015a740 con 0x7f267006ef60 2024-01-20T08:58:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.482+0000 7f267799b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267015acc0 con 0x7f2670159710 2024-01-20T08:58:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.483+0000 7f2675737640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 0x7f267015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.483+0000 7f2675f38640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2670159710 0x7f2670147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.483+0000 7f2674f36640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2670153800 0x7f26701e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.483+0000 7f2675737640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 0x7f267015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49908/0 (socket says 172.21.15.87:49908) 2024-01-20T08:58:52.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.483+0000 7f2675737640 1 -- 172.21.15.87:0/1276366558 learned_addr learned my addr 172.21.15.87:0/1276366558 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.486+0000 7f2675737640 1 -- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2670159710 msgr2=0x7f2670147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.486+0000 7f2675737640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2670159710 0x7f2670147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.486+0000 7f2675737640 1 -- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2670153800 msgr2=0x7f26701e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.486+0000 7f2675737640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2670153800 0x7f26701e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.486+0000 7f2675737640 1 -- 172.21.15.87:0/1276366558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c009400 con 0x7f267006ef60 2024-01-20T08:58:52.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.489+0000 7f2675737640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 0x7f267015b460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f266000c400 tx=0x7f266000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:52.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.492+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660005ce0 con 0x7f267006ef60 2024-01-20T08:58:52.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.492+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660005e40 con 0x7f267006ef60 2024-01-20T08:58:52.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.492+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267015a9e0 con 0x7f267006ef60 2024-01-20T08:58:52.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.492+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600072d0 con 0x7f267006ef60 2024-01-20T08:58:52.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.492+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2670150a30 con 0x7f267006ef60 2024-01-20T08:58:52.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.493+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2670074290 con 0x7f267006ef60 2024-01-20T08:58:52.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.511+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f26600074f0 con 0x7f267006ef60 2024-01-20T08:58:52.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.511+0000 7f2665ffb640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f265406af20 0x7f265406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.512+0000 7f2674f36640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f265406af20 0x7f265406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.512+0000 7f2674f36640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f265406af20 0x7f265406d3e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f266c0097d0 tx=0x7f266c03c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:52.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.740+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f2660057ca0 con 0x7f267006ef60 2024-01-20T08:58:52.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.750+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2660053fe0 con 0x7f267006ef60 2024-01-20T08:58:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.923+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f26700cacc0 con 0x7f267006ef60 2024-01-20T08:58:52.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.929+0000 7f2665ffb640 1 -- 172.21.15.87:0/1276366558 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2660017020 con 0x7f267006ef60 2024-01-20T08:58:52.930 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-01-20T08:58:52.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f265406af20 msgr2=0x7f265406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f265406af20 0x7f265406d3e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f266c0097d0 tx=0x7f266c03c040 comp rx=0 tx=0).stop 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 msgr2=0x7f267015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 0x7f267015b460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f266000c400 tx=0x7f266000c8c0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 shutdown_connections 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f265406af20 0x7f265406d3e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2670159710 0x7f2670147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2670153800 0x7f26701e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 --2- 172.21.15.87:0/1276366558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f267006ef60 0x7f267015b460 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 >> 172.21.15.87:0/1276366558 conn(0x7f2670070de0 msgr2=0x7f26700c7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 shutdown_connections 2024-01-20T08:58:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:52.930+0000 7f267799b640 1 -- 172.21.15.87:0/1276366558 wait complete. 2024-01-20T08:58:52.944 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-20T08:58:53.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.042+0000 7f7357725640 1 Processor -- start 2024-01-20T08:58:53.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.042+0000 7f7357725640 1 -- start start 2024-01-20T08:58:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f7357725640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f735014cc40 0x7f7350156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f7357725640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f7350161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f7357725640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7350156720 con 0x7f73500bfe70 2024-01-20T08:58:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f7357725640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7350072b30 con 0x7f735014cc40 2024-01-20T08:58:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f7357725640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73500c19c0 con 0x7f735015ec50 2024-01-20T08:58:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f7355cc2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f7350161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:53.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.043+0000 7f73554c1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f735014cc40 0x7f7350156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:53.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.044+0000 7f7355cc2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f7350161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:34732/0 (socket says 172.21.15.87:34732) 2024-01-20T08:58:53.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.044+0000 7f7355cc2640 1 -- 172.21.15.87:0/1698275701 learned_addr learned my addr 172.21.15.87:0/1698275701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:53.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.046+0000 7f7355cc2640 1 -- 172.21.15.87:0/1698275701 >> v2:172.21.15.116:3300/0 conn(0x7f735014cc40 msgr2=0x7f7350156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:58:53.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.046+0000 7f7355cc2640 1 --2- 172.21.15.87:0/1698275701 >> v2:172.21.15.116:3300/0 conn(0x7f735014cc40 0x7f7350156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.046+0000 7f7355cc2640 1 -- 172.21.15.87:0/1698275701 >> v1:172.21.15.116:6789/0 conn(0x7f73500bfe70 legacy=0x7f73500c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:53.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.046+0000 7f7355cc2640 1 -- 172.21.15.87:0/1698275701 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73501563f0 con 0x7f735015ec50 2024-01-20T08:58:53.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.048+0000 7f7355cc2640 1 --2- 172.21.15.87:0/1698275701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f7350161040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7348009e50 tx=0x7f734802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d9c5f3693f2f0e server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:53.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7347fff640 1 -- 172.21.15.87:0/1698275701 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734802cb90 con 0x7f735015ec50 2024-01-20T08:58:53.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7347fff640 1 -- 172.21.15.87:0/1698275701 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734802ccf0 con 0x7f735015ec50 2024-01-20T08:58:53.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7347fff640 1 -- 172.21.15.87:0/1698275701 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348034610 con 0x7f735015ec50 2024-01-20T08:58:53.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 -- 172.21.15.87:0/1698275701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 msgr2=0x7f7350161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 --2- 172.21.15.87:0/1698275701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f7350161040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7348009e50 tx=0x7f734802bff0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 -- 172.21.15.87:0/1698275701 shutdown_connections 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 --2- 172.21.15.87:0/1698275701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f7350161040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 --2- 172.21.15.87:0/1698275701 >> v2:172.21.15.116:3300/0 conn(0x7f735014cc40 0x7f7350156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 -- 172.21.15.87:0/1698275701 >> 172.21.15.87:0/1698275701 conn(0x7f735006a9a0 msgr2=0x7f735006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 -- 172.21.15.87:0/1698275701 shutdown_connections 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.052+0000 7f7357725640 1 -- 172.21.15.87:0/1698275701 wait complete. 2024-01-20T08:58:53.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.053+0000 7f7357725640 1 Processor -- start 2024-01-20T08:58:53.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 -- start start 2024-01-20T08:58:53.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73500bfe70 0x7f7350149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:53.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 0x7f73500ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:53.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f735015d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:53.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735015d500 con 0x7f73500bfe70 2024-01-20T08:58:53.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73500cbab0 con 0x7f735014cc40 2024-01-20T08:58:53.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.054+0000 7f7357725640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73500cb7d0 con 0x7f735015ec50 2024-01-20T08:58:53.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.055+0000 7f7354cc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 0x7f73500ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:53.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.055+0000 7f7355cc2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f735015d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:53.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.055+0000 7f73554c1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73500bfe70 0x7f7350149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:53.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.055+0000 7f7354cc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 0x7f73500ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:49932/0 (socket says 172.21.15.87:49932) 2024-01-20T08:58:53.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.055+0000 7f7354cc0640 1 -- 172.21.15.87:0/3022007118 learned_addr learned my addr 172.21.15.87:0/3022007118 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:58:53.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.058+0000 7f7354cc0640 1 -- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 msgr2=0x7f735015d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:53.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.058+0000 7f7354cc0640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f735015d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.058+0000 7f7354cc0640 1 -- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73500bfe70 msgr2=0x7f7350149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:53.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.058+0000 7f7354cc0640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73500bfe70 0x7f7350149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.058+0000 7f7354cc0640 1 -- 172.21.15.87:0/3022007118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7348009b30 con 0x7f735014cc40 2024-01-20T08:58:53.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.061+0000 7f7354cc0640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 0x7f73500ca8a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f734000c400 tx=0x7f734000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:53.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.065+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73400052d0 con 0x7f735014cc40 2024-01-20T08:58:53.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.065+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7340005450 con 0x7f735014cc40 2024-01-20T08:58:53.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.065+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340007260 con 0x7f735014cc40 2024-01-20T08:58:53.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.065+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73500c96e0 con 0x7f735014cc40 2024-01-20T08:58:53.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.065+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73500c9ca0 con 0x7f735014cc40 2024-01-20T08:58:53.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.066+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735014c780 con 0x7f735014cc40 2024-01-20T08:58:53.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.085+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f73400073c0 con 0x7f735014cc40 2024-01-20T08:58:53.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.085+0000 7f7345ffb640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f733406af00 0x7f733406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:58:53.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.085+0000 7f73554c1640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f733406af00 0x7f733406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:58:53.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.086+0000 7f73554c1640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f733406af00 0x7f733406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f734c009b40 tx=0x7f734c009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:58:53.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.315+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7340057ed0 con 0x7f735014cc40 2024-01-20T08:58:53.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.324+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7340054210 con 0x7f735014cc40 2024-01-20T08:58:53.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.529+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f735015d670 con 0x7f735014cc40 2024-01-20T08:58:53.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.534+0000 7f7345ffb640 1 -- 172.21.15.87:0/3022007118 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7340020080 con 0x7f735014cc40 2024-01-20T08:58:53.535 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-01-20T08:58:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f733406af00 msgr2=0x7f733406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f733406af00 0x7f733406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f734c009b40 tx=0x7f734c009190 comp rx=0 tx=0).stop 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 msgr2=0x7f73500ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 0x7f73500ca8a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f734000c400 tx=0x7f734000c8c0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 shutdown_connections 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f733406af00 0x7f733406d3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f735015ec50 0x7f735015d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f735014cc40 0x7f73500ca8a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 --2- 172.21.15.87:0/3022007118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73500bfe70 0x7f7350149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 >> 172.21.15.87:0/3022007118 conn(0x7f735006a9a0 msgr2=0x7f735014bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 shutdown_connections 2024-01-20T08:58:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:58:53.535+0000 7f7357725640 1 -- 172.21.15.87:0/3022007118 wait complete. 2024-01-20T08:58:53.548 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-01-20T08:58:53.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:58:53.548 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-20T08:58:53.576 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T08:58:53.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[47020]: fuse finished with error 0 and tester_r 0 2024-01-20T08:58:54.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:==00:00:26:50.801 47020== 2024-01-20T08:58:54.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:==00:00:26:50.801 47020== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T08:58:54.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:daemon-helper: command failed with exit status 42 2024-01-20T08:58:59.678 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T08:58:59.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T08:58:59.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:58:59.678 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-20T08:58:59.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:58:59.695 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-01-20T08:58:59.751 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-20T08:58:59.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:58:59.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-20T08:58:59.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-20T08:58:59.820 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-01-20T08:58:59.917 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-20T08:58:59.926 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-20T08:58:59.926 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:59:00.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.029+0000 7ff0734d3640 1 Processor -- start 2024-01-20T08:59:00.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.029+0000 7ff0734d3640 1 -- start start 2024-01-20T08:59:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.030+0000 7ff0734d3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff06c14fde0 0x7ff06c0727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.030+0000 7ff0734d3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff06c0721b0 0x7ff06c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.030+0000 7ff0734d3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c156070 con 0x7ff06c1590e0 2024-01-20T08:59:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.030+0000 7ff0734d3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c155af0 con 0x7ff06c0721b0 2024-01-20T08:59:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.030+0000 7ff0734d3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c06eb90 con 0x7ff06c14fde0 2024-01-20T08:59:00.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.031+0000 7ff07126f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff06c14fde0 0x7ff06c0727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.031+0000 7ff07126f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff06c14fde0 0x7ff06c0727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58030/0 (socket says 172.21.15.87:58030) 2024-01-20T08:59:00.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.031+0000 7ff07126f640 1 -- 172.21.15.87:0/3234705562 learned_addr learned my addr 172.21.15.87:0/3234705562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.034+0000 7ff07126f640 1 -- 172.21.15.87:0/3234705562 >> v2:172.21.15.87:3300/0 conn(0x7ff06c0721b0 msgr2=0x7ff06c06e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.034+0000 7ff07126f640 1 --2- 172.21.15.87:0/3234705562 >> v2:172.21.15.87:3300/0 conn(0x7ff06c0721b0 0x7ff06c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.034+0000 7ff07126f640 1 -- 172.21.15.87:0/3234705562 >> v1:172.21.15.87:6789/0 conn(0x7ff06c1590e0 legacy=0x7ff06c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:00.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.034+0000 7ff07126f640 1 -- 172.21.15.87:0/3234705562 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c1501c0 con 0x7ff06c14fde0 2024-01-20T08:59:00.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.037+0000 7ff07126f640 1 --2- 172.21.15.87:0/3234705562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0727c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0c5ae0 tx=0x7ff05c02cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3632b3e89cb8b599 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:00.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff063fff640 1 -- 172.21.15.87:0/3234705562 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c02dc50 con 0x7ff06c14fde0 2024-01-20T08:59:00.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff063fff640 1 -- 172.21.15.87:0/3234705562 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c02ddb0 con 0x7ff06c14fde0 2024-01-20T08:59:00.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff063fff640 1 -- 172.21.15.87:0/3234705562 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035660 con 0x7ff06c14fde0 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 -- 172.21.15.87:0/3234705562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 msgr2=0x7ff06c0727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 --2- 172.21.15.87:0/3234705562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0727c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0c5ae0 tx=0x7ff05c02cea0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 -- 172.21.15.87:0/3234705562 shutdown_connections 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 --2- 172.21.15.87:0/3234705562 >> v2:172.21.15.87:3300/0 conn(0x7ff06c0721b0 0x7ff06c06e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 --2- 172.21.15.87:0/3234705562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0727c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 -- 172.21.15.87:0/3234705562 >> 172.21.15.87:0/3234705562 conn(0x7ff06c06d510 msgr2=0x7ff06c0663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 -- 172.21.15.87:0/3234705562 shutdown_connections 2024-01-20T08:59:00.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.042+0000 7ff0734d3640 1 -- 172.21.15.87:0/3234705562 wait complete. 2024-01-20T08:59:00.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 Processor -- start 2024-01-20T08:59:00.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 -- start start 2024-01-20T08:59:00.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 0x7ff06c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff06c1590e0 0x7ff06c14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c1ef300 con 0x7ff06c0721b0 2024-01-20T08:59:00.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c158f40 con 0x7ff06c14fde0 2024-01-20T08:59:00.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.044+0000 7ff0734d3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c157e40 con 0x7ff06c1590e0 2024-01-20T08:59:00.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.045+0000 7ff07126f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 0x7ff06c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.045+0000 7ff070a6e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.045+0000 7ff07126f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 0x7ff06c158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59988/0 (socket says 172.21.15.87:59988) 2024-01-20T08:59:00.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.045+0000 7ff071a70640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff06c1590e0 0x7ff06c14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.045+0000 7ff07126f640 1 -- 172.21.15.87:0/770205191 learned_addr learned my addr 172.21.15.87:0/770205191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:00.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.048+0000 7ff07126f640 1 -- 172.21.15.87:0/770205191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff06c1590e0 msgr2=0x7ff06c14a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.048+0000 7ff07126f640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff06c1590e0 0x7ff06c14a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.048+0000 7ff07126f640 1 -- 172.21.15.87:0/770205191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 msgr2=0x7ff06c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.048+0000 7ff07126f640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.048+0000 7ff07126f640 1 -- 172.21.15.87:0/770205191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c00a670 con 0x7ff06c0721b0 2024-01-20T08:59:00.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.051+0000 7ff07126f640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 0x7ff06c158a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02cee0 tx=0x7ff05c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.054+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c03e600 con 0x7ff06c0721b0 2024-01-20T08:59:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.054+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c03e760 con 0x7ff06c0721b0 2024-01-20T08:59:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.054+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c03eaf0 con 0x7ff06c0721b0 2024-01-20T08:59:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.054+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c14c710 con 0x7ff06c0721b0 2024-01-20T08:59:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.054+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c1ef7f0 con 0x7ff06c0721b0 2024-01-20T08:59:00.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.055+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c0c67e0 con 0x7ff06c0721b0 2024-01-20T08:59:00.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.296+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff05c03c040 con 0x7ff06c0721b0 2024-01-20T08:59:00.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.296+0000 7ff061ffb640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05006af20 0x7ff05006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.297+0000 7ff070a6e640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05006af20 0x7ff05006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.297+0000 7ff070a6e640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05006af20 0x7ff05006d3e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff06c1e8f70 tx=0x7ff068009110 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:00.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.311+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7ff05c07b960 con 0x7ff06c0721b0 2024-01-20T08:59:00.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.311+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff05c077ca0 con 0x7ff06c0721b0 2024-01-20T08:59:00.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.481+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff06c06f370 con 0x7ff05006af20 2024-01-20T08:59:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.491+0000 7ff061ffb640 1 -- 172.21.15.87:0/770205191 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371333 (secure 0 0 0) 0x7ff0680095e0 con 0x7ff05006af20 2024-01-20T08:59:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05006af20 msgr2=0x7ff05006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05006af20 0x7ff05006d3e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff06c1e8f70 tx=0x7ff068009110 comp rx=0 tx=0).stop 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 msgr2=0x7ff06c158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 0x7ff06c158a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02cee0 tx=0x7ff05c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 shutdown_connections 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05006af20 0x7ff05006d3e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff06c1590e0 0x7ff06c14a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff06c14fde0 0x7ff06c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 --2- 172.21.15.87:0/770205191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff06c0721b0 0x7ff06c158a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 >> 172.21.15.87:0/770205191 conn(0x7ff06c06d510 msgr2=0x7ff06c0c1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:00.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 shutdown_connections 2024-01-20T08:59:00.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.495+0000 7ff0734d3640 1 -- 172.21.15.87:0/770205191 wait complete. 2024-01-20T08:59:00.507 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":865,"stamp":"2024-01-20T08:58:58.651732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39627569,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":1083,"num_read_kb":1128,"num_write":12670,"num_write_kb":354190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14976,"ondisk_log_size":14976,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461136,"kb_used_data":79888,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754513584,"statfs":{"total":773094113280,"available":772621910016,"internally_reserved":0,"allocated":81805312,"data_stored":80194922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390319664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61367,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":41,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002969"},"pg_stats":[{"pgid":"2.1f","version":"27'609","reported_seq":641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:34.122861+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:58:34.122861+0000","last_peered":"2024-01-20T08:58:34.122861+0000","last_clean":"2024-01-20T08:58:34.122861+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:58:34.122861+0000","last_undegraded":"2024-01-20T08:58:34.122861+0000","last_fullsized":"2024-01-20T08:58:34.122861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"27'27","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808166+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:58:53.808166+0000","last_peered":"2024-01-20T08:58:53.808166+0000","last_clean":"2024-01-20T08:58:53.808166+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:58:53.808166+0000","last_undegraded":"2024-01-20T08:58:53.808166+0000","last_fullsized":"2024-01-20T08:58:53.808166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"27'162","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.884124+0000","last_change":"2024-01-20T08:31:44.811554+0000","last_active":"2024-01-20T08:58:24.884124+0000","last_peered":"2024-01-20T08:58:24.884124+0000","last_clean":"2024-01-20T08:58:24.884124+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:58:24.884124+0000","last_undegraded":"2024-01-20T08:58:24.884124+0000","last_fullsized":"2024-01-20T08:58:24.884124+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:31:34.560783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":10773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'696","reported_seq":728,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.571964+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:58:47.571964+0000","last_peered":"2024-01-20T08:58:47.571964+0000","last_clean":"2024-01-20T08:58:47.571964+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:58:47.571964+0000","last_undegraded":"2024-01-20T08:58:47.571964+0000","last_fullsized":"2024-01-20T08:58:47.571964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"27'37","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843652+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:58:53.843652+0000","last_peered":"2024-01-20T08:58:53.843652+0000","last_clean":"2024-01-20T08:58:53.843652+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:58:53.843652+0000","last_undegraded":"2024-01-20T08:58:53.843652+0000","last_fullsized":"2024-01-20T08:58:53.843652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"27'136","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.633246+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:58:22.633246+0000","last_peered":"2024-01-20T08:58:22.633246+0000","last_clean":"2024-01-20T08:58:22.633246+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:58:22.633246+0000","last_undegraded":"2024-01-20T08:58:22.633246+0000","last_fullsized":"2024-01-20T08:58:22.633246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'1352","reported_seq":1382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.534987+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:58:46.534987+0000","last_peered":"2024-01-20T08:58:46.534987+0000","last_clean":"2024-01-20T08:58:46.534987+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:58:46.534987+0000","last_undegraded":"2024-01-20T08:58:46.534987+0000","last_fullsized":"2024-01-20T08:58:46.534987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.883892+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:58:53.883892+0000","last_peered":"2024-01-20T08:58:53.883892+0000","last_clean":"2024-01-20T08:58:53.883892+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:58:53.883892+0000","last_undegraded":"2024-01-20T08:58:53.883892+0000","last_fullsized":"2024-01-20T08:58:53.883892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"27'128","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:20.331609+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:58:20.331609+0000","last_peered":"2024-01-20T08:58:20.331609+0000","last_clean":"2024-01-20T08:58:20.331609+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:58:20.331609+0000","last_undegraded":"2024-01-20T08:58:20.331609+0000","last_fullsized":"2024-01-20T08:58:20.331609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:37.699537+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:58:37.699537+0000","last_peered":"2024-01-20T08:58:37.699537+0000","last_clean":"2024-01-20T08:58:37.699537+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:58:37.699537+0000","last_undegraded":"2024-01-20T08:58:37.699537+0000","last_fullsized":"2024-01-20T08:58:37.699537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"27'47","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878405+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:58:53.878405+0000","last_peered":"2024-01-20T08:58:53.878405+0000","last_clean":"2024-01-20T08:58:53.878405+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:58:53.878405+0000","last_undegraded":"2024-01-20T08:58:53.878405+0000","last_fullsized":"2024-01-20T08:58:53.878405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"27'149","reported_seq":175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.165676+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:58:23.165676+0000","last_peered":"2024-01-20T08:58:23.165676+0000","last_clean":"2024-01-20T08:58:23.165676+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:58:23.165676+0000","last_undegraded":"2024-01-20T08:58:23.165676+0000","last_fullsized":"2024-01-20T08:58:23.165676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":7763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.884346+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:58:53.884346+0000","last_peered":"2024-01-20T08:58:53.884346+0000","last_clean":"2024-01-20T08:58:53.884346+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:58:53.884346+0000","last_undegraded":"2024-01-20T08:58:53.884346+0000","last_fullsized":"2024-01-20T08:58:53.884346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'593","reported_seq":624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.320730+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:58:40.320730+0000","last_peered":"2024-01-20T08:58:40.320730+0000","last_clean":"2024-01-20T08:58:40.320730+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:58:40.320730+0000","last_undegraded":"2024-01-20T08:58:40.320730+0000","last_fullsized":"2024-01-20T08:58:40.320730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":4387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'109","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.837239+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:58:24.837239+0000","last_peered":"2024-01-20T08:58:24.837239+0000","last_clean":"2024-01-20T08:58:24.837239+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:58:24.837239+0000","last_undegraded":"2024-01-20T08:58:24.837239+0000","last_fullsized":"2024-01-20T08:58:24.837239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.840307+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:58:53.840307+0000","last_peered":"2024-01-20T08:58:53.840307+0000","last_clean":"2024-01-20T08:58:53.840307+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:58:53.840307+0000","last_undegraded":"2024-01-20T08:58:53.840307+0000","last_fullsized":"2024-01-20T08:58:53.840307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294867+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:58:39.294867+0000","last_peered":"2024-01-20T08:58:39.294867+0000","last_clean":"2024-01-20T08:58:39.294867+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:58:39.294867+0000","last_undegraded":"2024-01-20T08:58:39.294867+0000","last_fullsized":"2024-01-20T08:58:39.294867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"27'149","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.869606+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:58:23.869606+0000","last_peered":"2024-01-20T08:58:23.869606+0000","last_clean":"2024-01-20T08:58:23.869606+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:58:23.869606+0000","last_undegraded":"2024-01-20T08:58:23.869606+0000","last_fullsized":"2024-01-20T08:58:23.869606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.926690+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:58:53.926690+0000","last_peered":"2024-01-20T08:58:53.926690+0000","last_clean":"2024-01-20T08:58:53.926690+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:58:53.926690+0000","last_undegraded":"2024-01-20T08:58:53.926690+0000","last_fullsized":"2024-01-20T08:58:53.926690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'13","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.531366+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:58:46.531366+0000","last_peered":"2024-01-20T08:58:46.531366+0000","last_clean":"2024-01-20T08:58:46.531366+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:58:46.531366+0000","last_undegraded":"2024-01-20T08:58:46.531366+0000","last_fullsized":"2024-01-20T08:58:46.531366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"27'121","reported_seq":148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.284930+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:58:23.284930+0000","last_peered":"2024-01-20T08:58:23.284930+0000","last_clean":"2024-01-20T08:58:23.284930+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:58:23.284930+0000","last_undegraded":"2024-01-20T08:58:23.284930+0000","last_fullsized":"2024-01-20T08:58:23.284930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'32","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.847454+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:58:25.847454+0000","last_peered":"2024-01-20T08:58:25.847454+0000","last_clean":"2024-01-20T08:58:25.847454+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:58:25.847454+0000","last_undegraded":"2024-01-20T08:58:25.847454+0000","last_fullsized":"2024-01-20T08:58:25.847454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:42.498428+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:58:42.498428+0000","last_peered":"2024-01-20T08:58:42.498428+0000","last_clean":"2024-01-20T08:58:42.498428+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:58:42.498428+0000","last_undegraded":"2024-01-20T08:58:42.498428+0000","last_fullsized":"2024-01-20T08:58:42.498428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'112","reported_seq":139,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:21.538368+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:58:21.538368+0000","last_peered":"2024-01-20T08:58:21.538368+0000","last_clean":"2024-01-20T08:58:21.538368+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:58:21.538368+0000","last_undegraded":"2024-01-20T08:58:21.538368+0000","last_fullsized":"2024-01-20T08:58:21.538368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928171+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:58:53.928171+0000","last_peered":"2024-01-20T08:58:53.928171+0000","last_clean":"2024-01-20T08:58:53.928171+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:58:53.928171+0000","last_undegraded":"2024-01-20T08:58:53.928171+0000","last_fullsized":"2024-01-20T08:58:53.928171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"27'12","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:44.519111+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:58:44.519111+0000","last_peered":"2024-01-20T08:58:44.519111+0000","last_clean":"2024-01-20T08:58:44.519111+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:58:44.519111+0000","last_undegraded":"2024-01-20T08:58:44.519111+0000","last_fullsized":"2024-01-20T08:58:44.519111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.165734+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:58:23.165734+0000","last_peered":"2024-01-20T08:58:23.165734+0000","last_clean":"2024-01-20T08:58:23.165734+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:58:23.165734+0000","last_undegraded":"2024-01-20T08:58:23.165734+0000","last_fullsized":"2024-01-20T08:58:23.165734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929778+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:58:53.929778+0000","last_peered":"2024-01-20T08:58:53.929778+0000","last_clean":"2024-01-20T08:58:53.929778+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:58:53.929778+0000","last_undegraded":"2024-01-20T08:58:53.929778+0000","last_fullsized":"2024-01-20T08:58:53.929778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'315","reported_seq":346,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.274403+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:58:47.274403+0000","last_peered":"2024-01-20T08:58:47.274403+0000","last_clean":"2024-01-20T08:58:47.274403+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:58:47.274403+0000","last_undegraded":"2024-01-20T08:58:47.274403+0000","last_fullsized":"2024-01-20T08:58:47.274403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.953625+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:58:23.953625+0000","last_peered":"2024-01-20T08:58:23.953625+0000","last_clean":"2024-01-20T08:58:23.953625+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:58:23.953625+0000","last_undegraded":"2024-01-20T08:58:23.953625+0000","last_fullsized":"2024-01-20T08:58:23.953625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":11573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.800916+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:58:53.800916+0000","last_peered":"2024-01-20T08:58:53.800916+0000","last_clean":"2024-01-20T08:58:53.800916+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:58:53.800916+0000","last_undegraded":"2024-01-20T08:58:53.800916+0000","last_fullsized":"2024-01-20T08:58:53.800916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.323865+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:58:39.323865+0000","last_peered":"2024-01-20T08:58:39.323865+0000","last_clean":"2024-01-20T08:58:39.323865+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:58:39.323865+0000","last_undegraded":"2024-01-20T08:58:39.323865+0000","last_fullsized":"2024-01-20T08:58:39.323865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"27'126","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.957683+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:58:22.957683+0000","last_peered":"2024-01-20T08:58:22.957683+0000","last_clean":"2024-01-20T08:58:22.957683+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:58:22.957683+0000","last_undegraded":"2024-01-20T08:58:22.957683+0000","last_fullsized":"2024-01-20T08:58:22.957683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892100+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:58:53.892100+0000","last_peered":"2024-01-20T08:58:53.892100+0000","last_clean":"2024-01-20T08:58:53.892100+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:58:53.892100+0000","last_undegraded":"2024-01-20T08:58:53.892100+0000","last_fullsized":"2024-01-20T08:58:53.892100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:43.516881+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:58:43.516881+0000","last_peered":"2024-01-20T08:58:43.516881+0000","last_clean":"2024-01-20T08:58:43.516881+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:58:43.516881+0000","last_undegraded":"2024-01-20T08:58:43.516881+0000","last_fullsized":"2024-01-20T08:58:43.516881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"27'181","reported_seq":209,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.853812+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:58:22.853812+0000","last_peered":"2024-01-20T08:58:22.853812+0000","last_clean":"2024-01-20T08:58:22.853812+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:58:22.853812+0000","last_undegraded":"2024-01-20T08:58:22.853812+0000","last_fullsized":"2024-01-20T08:58:22.853812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019865+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:58:54.019865+0000","last_peered":"2024-01-20T08:58:54.019865+0000","last_clean":"2024-01-20T08:58:54.019865+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:58:54.019865+0000","last_undegraded":"2024-01-20T08:58:54.019865+0000","last_fullsized":"2024-01-20T08:58:54.019865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"27'13","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.315491+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:58:40.315491+0000","last_peered":"2024-01-20T08:58:40.315491+0000","last_clean":"2024-01-20T08:58:40.315491+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:58:40.315491+0000","last_undegraded":"2024-01-20T08:58:40.315491+0000","last_fullsized":"2024-01-20T08:58:40.315491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'113","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.935521+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:58:23.935521+0000","last_peered":"2024-01-20T08:58:23.935521+0000","last_clean":"2024-01-20T08:58:23.935521+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:58:23.935521+0000","last_undegraded":"2024-01-20T08:58:23.935521+0000","last_fullsized":"2024-01-20T08:58:23.935521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808962+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:58:53.808962+0000","last_peered":"2024-01-20T08:58:53.808962+0000","last_clean":"2024-01-20T08:58:53.808962+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:58:53.808962+0000","last_undegraded":"2024-01-20T08:58:53.808962+0000","last_fullsized":"2024-01-20T08:58:53.808962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"27'978","reported_seq":1977,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:49.545029+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:58:49.545029+0000","last_peered":"2024-01-20T08:58:49.545029+0000","last_clean":"2024-01-20T08:58:49.545029+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:58:49.545029+0000","last_undegraded":"2024-01-20T08:58:49.545029+0000","last_fullsized":"2024-01-20T08:58:49.545029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":978,"log_dups_size":0,"ondisk_log_size":978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210232,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":976,"num_write_kb":1034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"27'135","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.812343+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:58:25.812343+0000","last_peered":"2024-01-20T08:58:25.812343+0000","last_clean":"2024-01-20T08:58:25.812343+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:58:25.812343+0000","last_undegraded":"2024-01-20T08:58:25.812343+0000","last_fullsized":"2024-01-20T08:58:25.812343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118,"num_write_kb":10261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"27'28","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804624+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:58:53.804624+0000","last_peered":"2024-01-20T08:58:53.804624+0000","last_clean":"2024-01-20T08:58:53.804624+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:58:53.804624+0000","last_undegraded":"2024-01-20T08:58:53.804624+0000","last_fullsized":"2024-01-20T08:58:53.804624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"27'429","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.345853+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:58:41.345853+0000","last_peered":"2024-01-20T08:58:41.345853+0000","last_clean":"2024-01-20T08:58:41.345853+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:58:41.345853+0000","last_undegraded":"2024-01-20T08:58:41.345853+0000","last_fullsized":"2024-01-20T08:58:41.345853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":4309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"27'129","reported_seq":156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.864992+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:58:23.864992+0000","last_peered":"2024-01-20T08:58:23.864992+0000","last_clean":"2024-01-20T08:58:23.864992+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:58:23.864992+0000","last_undegraded":"2024-01-20T08:58:23.864992+0000","last_fullsized":"2024-01-20T08:58:23.864992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881046+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:58:53.881046+0000","last_peered":"2024-01-20T08:58:53.881046+0000","last_clean":"2024-01-20T08:58:53.881046+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:58:53.881046+0000","last_undegraded":"2024-01-20T08:58:53.881046+0000","last_fullsized":"2024-01-20T08:58:53.881046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"27'9","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.105404+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:58:33.105404+0000","last_peered":"2024-01-20T08:58:33.105404+0000","last_clean":"2024-01-20T08:58:33.105404+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:58:33.105404+0000","last_undegraded":"2024-01-20T08:58:33.105404+0000","last_fullsized":"2024-01-20T08:58:33.105404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:56:51.793427+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:56:51.793427+0000","last_peered":"2024-01-20T08:56:51.793427+0000","last_clean":"2024-01-20T08:56:51.793427+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:56:51.793427+0000","last_undegraded":"2024-01-20T08:56:51.793427+0000","last_fullsized":"2024-01-20T08:56:51.793427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'156","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.838096+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:58:24.838096+0000","last_peered":"2024-01-20T08:58:24.838096+0000","last_clean":"2024-01-20T08:58:24.838096+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:58:24.838096+0000","last_undegraded":"2024-01-20T08:58:24.838096+0000","last_fullsized":"2024-01-20T08:58:24.838096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":9711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.880987+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:58:53.880987+0000","last_peered":"2024-01-20T08:58:53.880987+0000","last_clean":"2024-01-20T08:58:53.880987+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:58:53.880987+0000","last_undegraded":"2024-01-20T08:58:53.880987+0000","last_fullsized":"2024-01-20T08:58:53.880987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569355+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:58:45.569355+0000","last_peered":"2024-01-20T08:58:45.569355+0000","last_clean":"2024-01-20T08:58:45.569355+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:58:45.569355+0000","last_undegraded":"2024-01-20T08:58:45.569355+0000","last_fullsized":"2024-01-20T08:58:45.569355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"27'130","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.565862+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:58:23.565862+0000","last_peered":"2024-01-20T08:58:23.565862+0000","last_clean":"2024-01-20T08:58:23.565862+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:58:23.565862+0000","last_undegraded":"2024-01-20T08:58:23.565862+0000","last_fullsized":"2024-01-20T08:58:23.565862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":4819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'39","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.021114+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:58:54.021114+0000","last_peered":"2024-01-20T08:58:54.021114+0000","last_clean":"2024-01-20T08:58:54.021114+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:58:54.021114+0000","last_undegraded":"2024-01-20T08:58:54.021114+0000","last_fullsized":"2024-01-20T08:58:54.021114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.352940+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:58:41.352940+0000","last_peered":"2024-01-20T08:58:41.352940+0000","last_clean":"2024-01-20T08:58:41.352940+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:58:41.352940+0000","last_undegraded":"2024-01-20T08:58:41.352940+0000","last_fullsized":"2024-01-20T08:58:41.352940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"27'188","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.886971+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:58:24.886971+0000","last_peered":"2024-01-20T08:58:24.886971+0000","last_clean":"2024-01-20T08:58:24.886971+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:58:24.886971+0000","last_undegraded":"2024-01-20T08:58:24.886971+0000","last_fullsized":"2024-01-20T08:58:24.886971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'35","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.845305+0000","last_change":"2024-01-20T08:31:44.212803+0000","last_active":"2024-01-20T08:58:53.845305+0000","last_peered":"2024-01-20T08:58:53.845305+0000","last_clean":"2024-01-20T08:58:53.845305+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:58:53.845305+0000","last_undegraded":"2024-01-20T08:58:53.845305+0000","last_fullsized":"2024-01-20T08:58:53.845305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:29.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.295236+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:58:39.295236+0000","last_peered":"2024-01-20T08:58:39.295236+0000","last_clean":"2024-01-20T08:58:39.295236+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:58:39.295236+0000","last_undegraded":"2024-01-20T08:58:39.295236+0000","last_fullsized":"2024-01-20T08:58:39.295236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"27'145","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.862293+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:58:24.862293+0000","last_peered":"2024-01-20T08:58:24.862293+0000","last_clean":"2024-01-20T08:58:24.862293+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:58:24.862293+0000","last_undegraded":"2024-01-20T08:58:24.862293+0000","last_fullsized":"2024-01-20T08:58:24.862293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.893905+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:58:53.893905+0000","last_peered":"2024-01-20T08:58:53.893905+0000","last_clean":"2024-01-20T08:58:53.893905+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:58:53.893905+0000","last_undegraded":"2024-01-20T08:58:53.893905+0000","last_fullsized":"2024-01-20T08:58:53.893905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:12.697455+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:58:12.697455+0000","last_peered":"2024-01-20T08:58:12.697455+0000","last_clean":"2024-01-20T08:58:12.697455+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:58:12.697455+0000","last_undegraded":"2024-01-20T08:58:12.697455+0000","last_fullsized":"2024-01-20T08:58:12.697455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"27'160","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.545680+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:58:23.545680+0000","last_peered":"2024-01-20T08:58:23.545680+0000","last_clean":"2024-01-20T08:58:23.545680+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:58:23.545680+0000","last_undegraded":"2024-01-20T08:58:23.545680+0000","last_fullsized":"2024-01-20T08:58:23.545680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":10942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"27'35","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.882452+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:58:53.882452+0000","last_peered":"2024-01-20T08:58:53.882452+0000","last_clean":"2024-01-20T08:58:53.882452+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:58:53.882452+0000","last_undegraded":"2024-01-20T08:58:53.882452+0000","last_fullsized":"2024-01-20T08:58:53.882452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:36.115658+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:58:36.115658+0000","last_peered":"2024-01-20T08:58:36.115658+0000","last_clean":"2024-01-20T08:58:36.115658+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:58:36.115658+0000","last_undegraded":"2024-01-20T08:58:36.115658+0000","last_fullsized":"2024-01-20T08:58:36.115658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"27'142","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.848076+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:58:24.848076+0000","last_peered":"2024-01-20T08:58:24.848076+0000","last_clean":"2024-01-20T08:58:24.848076+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:58:24.848076+0000","last_undegraded":"2024-01-20T08:58:24.848076+0000","last_fullsized":"2024-01-20T08:58:24.848076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881441+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:58:53.881441+0000","last_peered":"2024-01-20T08:58:53.881441+0000","last_clean":"2024-01-20T08:58:53.881441+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:58:53.881441+0000","last_undegraded":"2024-01-20T08:58:53.881441+0000","last_fullsized":"2024-01-20T08:58:53.881441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"27'11","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.570427+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:58:47.570427+0000","last_peered":"2024-01-20T08:58:47.570427+0000","last_clean":"2024-01-20T08:58:47.570427+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:58:47.570427+0000","last_undegraded":"2024-01-20T08:58:47.570427+0000","last_fullsized":"2024-01-20T08:58:47.570427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1169,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"27'133","reported_seq":159,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.549367+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:58:22.549367+0000","last_peered":"2024-01-20T08:58:22.549367+0000","last_clean":"2024-01-20T08:58:22.549367+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:58:22.549367+0000","last_undegraded":"2024-01-20T08:58:22.549367+0000","last_fullsized":"2024-01-20T08:58:22.549367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875228+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:58:53.875228+0000","last_peered":"2024-01-20T08:58:53.875228+0000","last_clean":"2024-01-20T08:58:53.875228+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:58:53.875228+0000","last_undegraded":"2024-01-20T08:58:53.875228+0000","last_fullsized":"2024-01-20T08:58:53.875228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.531270+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:58:45.531270+0000","last_peered":"2024-01-20T08:58:45.531270+0000","last_clean":"2024-01-20T08:58:45.531270+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:58:45.531270+0000","last_undegraded":"2024-01-20T08:58:45.531270+0000","last_fullsized":"2024-01-20T08:58:45.531270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"27'116","reported_seq":142,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.966157+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:58:23.966157+0000","last_peered":"2024-01-20T08:58:23.966157+0000","last_clean":"2024-01-20T08:58:23.966157+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:58:23.966157+0000","last_undegraded":"2024-01-20T08:58:23.966157+0000","last_fullsized":"2024-01-20T08:58:23.966157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"27'40","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.803393+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:58:53.803393+0000","last_peered":"2024-01-20T08:58:53.803393+0000","last_clean":"2024-01-20T08:58:53.803393+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:58:53.803393+0000","last_undegraded":"2024-01-20T08:58:53.803393+0000","last_fullsized":"2024-01-20T08:58:53.803393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"27'111","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.634588+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:58:53.634588+0000","last_peered":"2024-01-20T08:58:53.634588+0000","last_clean":"2024-01-20T08:58:53.634588+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:58:53.634588+0000","last_undegraded":"2024-01-20T08:58:53.634588+0000","last_fullsized":"2024-01-20T08:58:53.634588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069576,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"27'126","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.892559+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:58:23.892559+0000","last_peered":"2024-01-20T08:58:23.892559+0000","last_clean":"2024-01-20T08:58:23.892559+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:58:23.892559+0000","last_undegraded":"2024-01-20T08:58:23.892559+0000","last_fullsized":"2024-01-20T08:58:23.892559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.017745+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:58:54.017745+0000","last_peered":"2024-01-20T08:58:54.017745+0000","last_clean":"2024-01-20T08:58:54.017745+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:58:54.017745+0000","last_undegraded":"2024-01-20T08:58:54.017745+0000","last_fullsized":"2024-01-20T08:58:54.017745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'833","reported_seq":863,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:29.921887+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:58:29.921887+0000","last_peered":"2024-01-20T08:58:29.921887+0000","last_clean":"2024-01-20T08:58:29.921887+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:58:29.921887+0000","last_undegraded":"2024-01-20T08:58:29.921887+0000","last_fullsized":"2024-01-20T08:58:29.921887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"27'135","reported_seq":161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.466040+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:58:22.466040+0000","last_peered":"2024-01-20T08:58:22.466040+0000","last_clean":"2024-01-20T08:58:22.466040+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:58:22.466040+0000","last_undegraded":"2024-01-20T08:58:22.466040+0000","last_fullsized":"2024-01-20T08:58:22.466040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881221+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:58:53.881221+0000","last_peered":"2024-01-20T08:58:53.881221+0000","last_clean":"2024-01-20T08:58:53.881221+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:58:53.881221+0000","last_undegraded":"2024-01-20T08:58:53.881221+0000","last_fullsized":"2024-01-20T08:58:53.881221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"27'652","reported_seq":682,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.580377+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:58:47.580377+0000","last_peered":"2024-01-20T08:58:47.580377+0000","last_clean":"2024-01-20T08:58:47.580377+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:58:47.580377+0000","last_undegraded":"2024-01-20T08:58:47.580377+0000","last_fullsized":"2024-01-20T08:58:47.580377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.790975+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:58:24.790975+0000","last_peered":"2024-01-20T08:58:24.790975+0000","last_clean":"2024-01-20T08:58:24.790975+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:58:24.790975+0000","last_undegraded":"2024-01-20T08:58:24.790975+0000","last_fullsized":"2024-01-20T08:58:24.790975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":23255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.833806+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:58:53.833806+0000","last_peered":"2024-01-20T08:58:53.833806+0000","last_clean":"2024-01-20T08:58:53.833806+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:58:53.833806+0000","last_undegraded":"2024-01-20T08:58:53.833806+0000","last_fullsized":"2024-01-20T08:58:53.833806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.102700+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:58:33.102700+0000","last_peered":"2024-01-20T08:58:33.102700+0000","last_clean":"2024-01-20T08:58:33.102700+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:58:33.102700+0000","last_undegraded":"2024-01-20T08:58:33.102700+0000","last_fullsized":"2024-01-20T08:58:33.102700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"27'124","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.864733+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:58:24.864733+0000","last_peered":"2024-01-20T08:58:24.864733+0000","last_clean":"2024-01-20T08:58:24.864733+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:58:24.864733+0000","last_undegraded":"2024-01-20T08:58:24.864733+0000","last_fullsized":"2024-01-20T08:58:24.864733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895614+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:58:53.895614+0000","last_peered":"2024-01-20T08:58:53.895614+0000","last_clean":"2024-01-20T08:58:53.895614+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:58:53.895614+0000","last_undegraded":"2024-01-20T08:58:53.895614+0000","last_fullsized":"2024-01-20T08:58:53.895614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.632949+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:58:53.632949+0000","last_peered":"2024-01-20T08:58:53.632949+0000","last_clean":"2024-01-20T08:58:53.632949+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:58:53.632949+0000","last_undegraded":"2024-01-20T08:58:53.632949+0000","last_fullsized":"2024-01-20T08:58:53.632949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.554720+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:58:22.554720+0000","last_peered":"2024-01-20T08:58:22.554720+0000","last_clean":"2024-01-20T08:58:22.554720+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:58:22.554720+0000","last_undegraded":"2024-01-20T08:58:22.554720+0000","last_fullsized":"2024-01-20T08:58:22.554720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"27'130","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.958134+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:58:23.958134+0000","last_peered":"2024-01-20T08:58:23.958134+0000","last_clean":"2024-01-20T08:58:23.958134+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:58:23.958134+0000","last_undegraded":"2024-01-20T08:58:23.958134+0000","last_fullsized":"2024-01-20T08:58:23.958134+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":8002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.932256+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:58:53.932256+0000","last_peered":"2024-01-20T08:58:53.932256+0000","last_clean":"2024-01-20T08:58:53.932256+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:58:53.932256+0000","last_undegraded":"2024-01-20T08:58:53.932256+0000","last_fullsized":"2024-01-20T08:58:53.932256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.360632+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:58:41.360632+0000","last_peered":"2024-01-20T08:58:41.360632+0000","last_clean":"2024-01-20T08:58:41.360632+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:58:41.360632+0000","last_undegraded":"2024-01-20T08:58:41.360632+0000","last_fullsized":"2024-01-20T08:58:41.360632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"27'143","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.710016+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:58:24.710016+0000","last_peered":"2024-01-20T08:58:24.710016+0000","last_clean":"2024-01-20T08:58:24.710016+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:58:24.710016+0000","last_undegraded":"2024-01-20T08:58:24.710016+0000","last_fullsized":"2024-01-20T08:58:24.710016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":18955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'41","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.841394+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:58:53.841394+0000","last_peered":"2024-01-20T08:58:53.841394+0000","last_clean":"2024-01-20T08:58:53.841394+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:58:53.841394+0000","last_undegraded":"2024-01-20T08:58:53.841394+0000","last_fullsized":"2024-01-20T08:58:53.841394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294844+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:58:39.294844+0000","last_peered":"2024-01-20T08:58:39.294844+0000","last_clean":"2024-01-20T08:58:39.294844+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:58:39.294844+0000","last_undegraded":"2024-01-20T08:58:39.294844+0000","last_fullsized":"2024-01-20T08:58:39.294844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"27'675","reported_seq":705,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.711307+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:58:39.711307+0000","last_peered":"2024-01-20T08:58:39.711307+0000","last_clean":"2024-01-20T08:58:39.711307+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:58:39.711307+0000","last_undegraded":"2024-01-20T08:58:39.711307+0000","last_fullsized":"2024-01-20T08:58:39.711307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929004+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:58:53.929004+0000","last_peered":"2024-01-20T08:58:53.929004+0000","last_clean":"2024-01-20T08:58:53.929004+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:58:53.929004+0000","last_undegraded":"2024-01-20T08:58:53.929004+0000","last_fullsized":"2024-01-20T08:58:53.929004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"27'89","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.877290+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:58:23.877290+0000","last_peered":"2024-01-20T08:58:23.877290+0000","last_clean":"2024-01-20T08:58:23.877290+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:58:23.877290+0000","last_undegraded":"2024-01-20T08:58:23.877290+0000","last_fullsized":"2024-01-20T08:58:23.877290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":9333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'689","reported_seq":719,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569769+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:58:45.569769+0000","last_peered":"2024-01-20T08:58:45.569769+0000","last_clean":"2024-01-20T08:58:45.569769+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:58:45.569769+0000","last_undegraded":"2024-01-20T08:58:45.569769+0000","last_fullsized":"2024-01-20T08:58:45.569769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"27'33","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888587+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:58:53.888587+0000","last_peered":"2024-01-20T08:58:53.888587+0000","last_clean":"2024-01-20T08:58:53.888587+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:58:53.888587+0000","last_undegraded":"2024-01-20T08:58:53.888587+0000","last_fullsized":"2024-01-20T08:58:53.888587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"27'147","reported_seq":173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.840471+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:58:24.840471+0000","last_peered":"2024-01-20T08:58:24.840471+0000","last_clean":"2024-01-20T08:58:24.840471+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:58:24.840471+0000","last_undegraded":"2024-01-20T08:58:24.840471+0000","last_fullsized":"2024-01-20T08:58:24.840471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":10430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892176+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:58:53.892176+0000","last_peered":"2024-01-20T08:58:53.892176+0000","last_clean":"2024-01-20T08:58:53.892176+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:58:53.892176+0000","last_undegraded":"2024-01-20T08:58:53.892176+0000","last_fullsized":"2024-01-20T08:58:53.892176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843335+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:58:53.843335+0000","last_peered":"2024-01-20T08:58:53.843335+0000","last_clean":"2024-01-20T08:58:53.843335+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:58:53.843335+0000","last_undegraded":"2024-01-20T08:58:53.843335+0000","last_fullsized":"2024-01-20T08:58:53.843335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"27'44","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875423+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:58:53.875423+0000","last_peered":"2024-01-20T08:58:53.875423+0000","last_clean":"2024-01-20T08:58:53.875423+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:58:53.875423+0000","last_undegraded":"2024-01-20T08:58:53.875423+0000","last_fullsized":"2024-01-20T08:58:53.875423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.839038+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:58:53.839038+0000","last_peered":"2024-01-20T08:58:53.839038+0000","last_clean":"2024-01-20T08:58:53.839038+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:58:53.839038+0000","last_undegraded":"2024-01-20T08:58:53.839038+0000","last_fullsized":"2024-01-20T08:58:53.839038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"27'27","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.807055+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:58:53.807055+0000","last_peered":"2024-01-20T08:58:53.807055+0000","last_clean":"2024-01-20T08:58:53.807055+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:58:53.807055+0000","last_undegraded":"2024-01-20T08:58:53.807055+0000","last_fullsized":"2024-01-20T08:58:53.807055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'31","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.802128+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:58:53.802128+0000","last_peered":"2024-01-20T08:58:53.802128+0000","last_clean":"2024-01-20T08:58:53.802128+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:58:53.802128+0000","last_undegraded":"2024-01-20T08:58:53.802128+0000","last_fullsized":"2024-01-20T08:58:53.802128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"27'44","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895227+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:58:53.895227+0000","last_peered":"2024-01-20T08:58:53.895227+0000","last_clean":"2024-01-20T08:58:53.895227+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:58:53.895227+0000","last_undegraded":"2024-01-20T08:58:53.895227+0000","last_fullsized":"2024-01-20T08:58:53.895227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808904+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:58:53.808904+0000","last_peered":"2024-01-20T08:58:53.808904+0000","last_clean":"2024-01-20T08:58:53.808904+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:58:53.808904+0000","last_undegraded":"2024-01-20T08:58:53.808904+0000","last_fullsized":"2024-01-20T08:58:53.808904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.871511+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:58:53.871511+0000","last_peered":"2024-01-20T08:58:53.871511+0000","last_clean":"2024-01-20T08:58:53.871511+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:58:53.871511+0000","last_undegraded":"2024-01-20T08:58:53.871511+0000","last_fullsized":"2024-01-20T08:58:53.871511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"27'42","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.837960+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:58:53.837960+0000","last_peered":"2024-01-20T08:58:53.837960+0000","last_clean":"2024-01-20T08:58:53.837960+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:58:53.837960+0000","last_undegraded":"2024-01-20T08:58:53.837960+0000","last_fullsized":"2024-01-20T08:58:53.837960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"27'38","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888654+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:58:53.888654+0000","last_peered":"2024-01-20T08:58:53.888654+0000","last_clean":"2024-01-20T08:58:53.888654+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:58:53.888654+0000","last_undegraded":"2024-01-20T08:58:53.888654+0000","last_fullsized":"2024-01-20T08:58:53.888654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'47","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928357+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:58:53.928357+0000","last_peered":"2024-01-20T08:58:53.928357+0000","last_clean":"2024-01-20T08:58:53.928357+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:58:53.928357+0000","last_undegraded":"2024-01-20T08:58:53.928357+0000","last_fullsized":"2024-01-20T08:58:53.928357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.021424+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:58:54.021424+0000","last_peered":"2024-01-20T08:58:54.021424+0000","last_clean":"2024-01-20T08:58:54.021424+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:58:54.021424+0000","last_undegraded":"2024-01-20T08:58:54.021424+0000","last_fullsized":"2024-01-20T08:58:54.021424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878349+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:58:53.878349+0000","last_peered":"2024-01-20T08:58:53.878349+0000","last_clean":"2024-01-20T08:58:53.878349+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:58:53.878349+0000","last_undegraded":"2024-01-20T08:58:53.878349+0000","last_fullsized":"2024-01-20T08:58:53.878349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"27'45","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.846302+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:58:53.846302+0000","last_peered":"2024-01-20T08:58:53.846302+0000","last_clean":"2024-01-20T08:58:53.846302+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:58:53.846302+0000","last_undegraded":"2024-01-20T08:58:53.846302+0000","last_fullsized":"2024-01-20T08:58:53.846302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"27'38","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.870937+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:58:53.870937+0000","last_peered":"2024-01-20T08:58:53.870937+0000","last_clean":"2024-01-20T08:58:53.870937+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:58:53.870937+0000","last_undegraded":"2024-01-20T08:58:53.870937+0000","last_fullsized":"2024-01-20T08:58:53.870937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881740+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:58:53.881740+0000","last_peered":"2024-01-20T08:58:53.881740+0000","last_clean":"2024-01-20T08:58:53.881740+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:58:53.881740+0000","last_undegraded":"2024-01-20T08:58:53.881740+0000","last_fullsized":"2024-01-20T08:58:53.881740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.834894+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:58:53.834894+0000","last_peered":"2024-01-20T08:58:53.834894+0000","last_clean":"2024-01-20T08:58:53.834894+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:58:53.834894+0000","last_undegraded":"2024-01-20T08:58:53.834894+0000","last_fullsized":"2024-01-20T08:58:53.834894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804360+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:58:53.804360+0000","last_peered":"2024-01-20T08:58:53.804360+0000","last_clean":"2024-01-20T08:58:53.804360+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:58:53.804360+0000","last_undegraded":"2024-01-20T08:58:53.804360+0000","last_fullsized":"2024-01-20T08:58:53.804360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806862+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:58:53.806862+0000","last_peered":"2024-01-20T08:58:53.806862+0000","last_clean":"2024-01-20T08:58:53.806862+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:58:53.806862+0000","last_undegraded":"2024-01-20T08:58:53.806862+0000","last_fullsized":"2024-01-20T08:58:53.806862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023455+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:58:54.023455+0000","last_peered":"2024-01-20T08:58:54.023455+0000","last_clean":"2024-01-20T08:58:54.023455+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:58:54.023455+0000","last_undegraded":"2024-01-20T08:58:54.023455+0000","last_fullsized":"2024-01-20T08:58:54.023455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022954+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:58:54.022954+0000","last_peered":"2024-01-20T08:58:54.022954+0000","last_clean":"2024-01-20T08:58:54.022954+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:58:54.022954+0000","last_undegraded":"2024-01-20T08:58:54.022954+0000","last_fullsized":"2024-01-20T08:58:54.022954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023780+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:58:54.023780+0000","last_peered":"2024-01-20T08:58:54.023780+0000","last_clean":"2024-01-20T08:58:54.023780+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:58:54.023780+0000","last_undegraded":"2024-01-20T08:58:54.023780+0000","last_fullsized":"2024-01-20T08:58:54.023780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.889696+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:58:53.889696+0000","last_peered":"2024-01-20T08:58:53.889696+0000","last_clean":"2024-01-20T08:58:53.889696+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:58:53.889696+0000","last_undegraded":"2024-01-20T08:58:53.889696+0000","last_fullsized":"2024-01-20T08:58:53.889696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843441+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:58:53.843441+0000","last_peered":"2024-01-20T08:58:53.843441+0000","last_clean":"2024-01-20T08:58:53.843441+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:58:53.843441+0000","last_undegraded":"2024-01-20T08:58:53.843441+0000","last_fullsized":"2024-01-20T08:58:53.843441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.017192+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:58:54.017192+0000","last_peered":"2024-01-20T08:58:54.017192+0000","last_clean":"2024-01-20T08:58:54.017192+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:58:54.017192+0000","last_undegraded":"2024-01-20T08:58:54.017192+0000","last_fullsized":"2024-01-20T08:58:54.017192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892312+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:58:53.892312+0000","last_peered":"2024-01-20T08:58:53.892312+0000","last_clean":"2024-01-20T08:58:53.892312+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:58:53.892312+0000","last_undegraded":"2024-01-20T08:58:53.892312+0000","last_fullsized":"2024-01-20T08:58:53.892312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"27'30","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019038+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:58:54.019038+0000","last_peered":"2024-01-20T08:58:54.019038+0000","last_clean":"2024-01-20T08:58:54.019038+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:58:54.019038+0000","last_undegraded":"2024-01-20T08:58:54.019038+0000","last_fullsized":"2024-01-20T08:58:54.019038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'25","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892157+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:58:53.892157+0000","last_peered":"2024-01-20T08:58:53.892157+0000","last_clean":"2024-01-20T08:58:53.892157+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:58:53.892157+0000","last_undegraded":"2024-01-20T08:58:53.892157+0000","last_fullsized":"2024-01-20T08:58:53.892157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'41","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.842487+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:58:53.842487+0000","last_peered":"2024-01-20T08:58:53.842487+0000","last_clean":"2024-01-20T08:58:53.842487+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:58:53.842487+0000","last_undegraded":"2024-01-20T08:58:53.842487+0000","last_fullsized":"2024-01-20T08:58:53.842487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806164+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:58:53.806164+0000","last_peered":"2024-01-20T08:58:53.806164+0000","last_clean":"2024-01-20T08:58:53.806164+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:58:53.806164+0000","last_undegraded":"2024-01-20T08:58:53.806164+0000","last_fullsized":"2024-01-20T08:58:53.806164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"27'26","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022116+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:58:54.022116+0000","last_peered":"2024-01-20T08:58:54.022116+0000","last_clean":"2024-01-20T08:58:54.022116+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:58:54.022116+0000","last_undegraded":"2024-01-20T08:58:54.022116+0000","last_fullsized":"2024-01-20T08:58:54.022116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":308552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4420,"ondisk_log_size":4420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39037201,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":8593,"num_write_kb":43222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78323712,"data_stored":78074402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8116,"ondisk_log_size":8116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509784,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39732,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332108,"statfs":{"total":96636764160,"available":96596078592,"internally_reserved":0,"allocated":4501504,"data_stored":4315319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46999999999999997,"15min":0.45300000000000001},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.23899999999999999},"max":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.25800000000000001},"max":{"1min":0.60999999999999999,"5min":0.66200000000000003,"15min":0.81999999999999995},"last":0.46999999999999997}]},{"osd":1,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.67900000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28999999999999998},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.67500000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.70899999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.628,"5min":0.68500000000000005,"15min":0.72699999999999998},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.30299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.74199999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44800000000000001,"15min":0.45400000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.23799999999999999},"max":{"1min":0.63500000000000001,"5min":0.67400000000000004,"15min":0.76700000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75800000000000001,"15min":0.876},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.81000000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.79400000000000004},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.27600000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.90900000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"last":0.54400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424509784,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67036,"kb_used_data":13652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94304804,"statfs":{"total":96636764160,"available":96568119296,"internally_reserved":0,"allocated":13979648,"data_stored":13773597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.29099999999999998},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17199999999999999},"max":{"1min":0.41899999999999998,"5min":0.441,"15min":0.72599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.19500000000000001,"15min":0.19400000000000001},"max":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.58799999999999997},"last":0.28999999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.503,"15min":0.61799999999999999},"last":0.315},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.216,"15min":0.21099999999999999},"max":{"1min":0.39500000000000002,"5min":0.51000000000000001,"15min":0.79600000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.69499999999999995},"last":0.37}]},{"osd":4,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.72599999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.77600000000000002,"15min":0.91400000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.48499999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.78500000000000003},"last":0.497}]}]},{"osd":5,"up_from":14,"seq":60129542486,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70924,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300916,"statfs":{"total":96636764160,"available":96564137984,"internally_reserved":0,"allocated":13701120,"data_stored":13502280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"max":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.69599999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72099999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.66000000000000003},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.72899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.76700000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.80900000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44,"15min":0.439},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48399999999999999,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.26500000000000001},"max":{"1min":0.56799999999999995,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.81599999999999995},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.80300000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59399999999999997,"5min":0.76200000000000001,"15min":0.85799999999999998},"last":0.44400000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542485,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66068,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305772,"statfs":{"total":96636764160,"available":96569110528,"internally_reserved":0,"allocated":8728576,"data_stored":8508591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.61299999999999999,"15min":0.70199999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.629,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.31}]},{"osd":2,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.31,"15min":0.26500000000000001},"max":{"1min":0.51400000000000001,"5min":0.69599999999999995,"15min":0.73299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.315,"5min":0.31,"15min":0.28599999999999998},"max":{"1min":0.54700000000000004,"5min":0.622,"15min":0.66700000000000004},"last":0.38800000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57799999999999996,"5min":0.71899999999999997,"15min":0.73999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.254},"max":{"1min":0.61899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.26000000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.59099999999999997,"5min":0.75700000000000001,"15min":0.78400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":0.75600000000000001},"last":0.27200000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66000000000000003,"5min":0.78300000000000003,"15min":0.79100000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":0.71699999999999997,"15min":0.79000000000000004},"last":0.28899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542486,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43824,"kb_used_data":8488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328016,"statfs":{"total":96636764160,"available":96591888384,"internally_reserved":0,"allocated":8691712,"data_stored":8506821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.45800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29999999999999999,"15min":0.307},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":0.38100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.26300000000000001}]},{"osd":1,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.502,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.308}]},{"osd":3,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33000000000000002,"15min":0.33200000000000002},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.378,"5min":0.55900000000000005,"15min":0.60799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34599999999999997}]},{"osd":4,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.71399999999999997,"15min":0.82099999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.376}]},{"osd":5,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.86699999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.80500000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.69999999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.76000000000000001,"15min":0.79300000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.36399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85299999999999998},"last":0.439}]}]},{"osd":1,"up_from":14,"seq":60129542487,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9289728,"data_stored":9097863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30099999999999999,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.55700000000000005,"15min":0.59499999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.32000000000000001,"15min":0.31},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20000000000000001},"max":{"1min":0.40600000000000003,"5min":0.45500000000000002,"15min":0.54400000000000004},"last":0.307}]},{"osd":2,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.308,"15min":0.314},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.35399999999999998,"5min":0.47999999999999998,"15min":0.49199999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":0.45800000000000002,"5min":0.624,"15min":0.624},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.59399999999999997,"15min":0.63},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.44700000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.27900000000000003},"max":{"1min":0.80700000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39300000000000002}]},{"osd":5,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.876},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":0.873,"15min":0.873},"last":0.49199999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46400000000000002}]}]},{"osd":7,"up_from":13,"seq":55834575189,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63212,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308628,"statfs":{"total":96636764160,"available":96572035072,"internally_reserved":0,"allocated":5804032,"data_stored":5592459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.44},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"last":0.504}]},{"osd":2,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.68999999999999995},"last":0.49299999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.54200000000000004,"5min":0.65100000000000002,"15min":0.73499999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.37,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.78000000000000003},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23000000000000001},"max":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.75800000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26600000000000001},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.79900000000000004},"last":0.35199999999999998}]},{"osd":5,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"last":0.317},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.76400000000000001,"15min":0.77800000000000002},"last":0.38200000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.371},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.42499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575189,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61772,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310068,"statfs":{"total":96636764160,"available":96573509632,"internally_reserved":0,"allocated":17108992,"data_stored":16897992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.505,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.31,"15min":0.30399999999999999},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.41699999999999998,"5min":0.47599999999999998,"15min":0.51200000000000001},"last":0.30599999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47199999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.42299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.32600000000000001}]},{"osd":2,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44700000000000001,"5min":0.625,"15min":0.625},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.373,"15min":0.371},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.434,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.375}]},{"osd":4,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.78500000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.71399999999999997,"5min":0.80000000000000004,"15min":0.85499999999999998},"last":0.42999999999999999}]},{"osd":5,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.76100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.437}]},{"osd":6,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.34000000000000002},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44900000000000001}]},{"osd":7,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13692928,"data_stored":13656124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8409088,"data_stored":8390022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":8389348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16822272,"data_stored":16780519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8441856,"data_stored":8391118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12820480,"data_stored":12794439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4214784,"data_stored":4197846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":5474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:59:00.513 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:59:00.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.648+0000 7f7c019a9640 1 Processor -- start 2024-01-20T08:59:00.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.648+0000 7f7c019a9640 1 -- start start 2024-01-20T08:59:00.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.649+0000 7f7c019a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.649+0000 7f7c019a9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7bfc06bc00 0x7f7bfc159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.649+0000 7f7c019a9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc0c9a00 con 0x7f7bfc0c92f0 2024-01-20T08:59:00.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.649+0000 7f7c019a9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc1598c0 con 0x7f7bfc06bc00 2024-01-20T08:59:00.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.649+0000 7f7c019a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0c9cc0 con 0x7f7bfc153920 2024-01-20T08:59:00.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.650+0000 7f7bfaffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.650+0000 7f7bfb7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7bfc06bc00 0x7f7bfc159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.650+0000 7f7bfaffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc06b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50104/0 (socket says 172.21.15.87:50104) 2024-01-20T08:59:00.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.650+0000 7f7bfaffd640 1 -- 172.21.15.87:0/163258631 learned_addr learned my addr 172.21.15.87:0/163258631 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:00.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.650+0000 7f7bfa7fc640 1 --1- 172.21.15.87:0/163258631 >> v1:172.21.15.116:6789/0 conn(0x7f7bfc0c92f0 0x7f7bfc0c7a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:41978/0 (socket says 172.21.15.87:41978) 2024-01-20T08:59:00.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.653+0000 7f7bfaffd640 1 -- 172.21.15.87:0/163258631 >> v2:172.21.15.116:3300/0 conn(0x7f7bfc06bc00 msgr2=0x7f7bfc159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:00.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.653+0000 7f7bfaffd640 1 --2- 172.21.15.87:0/163258631 >> v2:172.21.15.116:3300/0 conn(0x7f7bfc06bc00 0x7f7bfc159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.653+0000 7f7bfaffd640 1 -- 172.21.15.87:0/163258631 >> v1:172.21.15.116:6789/0 conn(0x7f7bfc0c92f0 legacy=0x7f7bfc0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.653+0000 7f7bfaffd640 1 -- 172.21.15.87:0/163258631 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc0c9fa0 con 0x7f7bfc153920 2024-01-20T08:59:00.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.655+0000 7f7bfaffd640 1 --2- 172.21.15.87:0/163258631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc06b6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009e50 tx=0x7f7bec02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b702d5815b72c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:00.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.659+0000 7f7bf9ffb640 1 -- 172.21.15.87:0/163258631 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec02cb90 con 0x7f7bfc153920 2024-01-20T08:59:00.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.659+0000 7f7bf9ffb640 1 -- 172.21.15.87:0/163258631 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec02ccf0 con 0x7f7bfc153920 2024-01-20T08:59:00.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.659+0000 7f7bf9ffb640 1 -- 172.21.15.87:0/163258631 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec034610 con 0x7f7bfc153920 2024-01-20T08:59:00.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 -- 172.21.15.87:0/163258631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 msgr2=0x7f7bfc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 --2- 172.21.15.87:0/163258631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc06b6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009e50 tx=0x7f7bec02bff0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 -- 172.21.15.87:0/163258631 shutdown_connections 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 --2- 172.21.15.87:0/163258631 >> v2:172.21.15.116:3300/0 conn(0x7f7bfc06bc00 0x7f7bfc159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 --2- 172.21.15.87:0/163258631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc06b6c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 -- 172.21.15.87:0/163258631 >> 172.21.15.87:0/163258631 conn(0x7f7bfc06a5b0 msgr2=0x7f7bfc06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 -- 172.21.15.87:0/163258631 shutdown_connections 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.660+0000 7f7c019a9640 1 -- 172.21.15.87:0/163258631 wait complete. 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.661+0000 7f7c019a9640 1 Processor -- start 2024-01-20T08:59:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.661+0000 7f7c019a9640 1 -- start start 2024-01-20T08:59:00.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.662+0000 7f7c019a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bfc06bc00 0x7f7bfc1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.662+0000 7f7c019a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.662+0000 7f7c019a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bfc148590 0x7f7bfc1472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.662+0000 7f7c019a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc143c10 con 0x7f7bfc148590 2024-01-20T08:59:00.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.662+0000 7f7c019a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc1468c0 con 0x7f7bfc06bc00 2024-01-20T08:59:00.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.662+0000 7f7c019a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc149890 con 0x7f7bfc153920 2024-01-20T08:59:00.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.663+0000 7f7bfa7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.663+0000 7f7bfaffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bfc06bc00 0x7f7bfc1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.663+0000 7f7bfa7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50118/0 (socket says 172.21.15.87:50118) 2024-01-20T08:59:00.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.663+0000 7f7bfa7fc640 1 -- 172.21.15.87:0/3602247367 learned_addr learned my addr 172.21.15.87:0/3602247367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:00.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.667+0000 7f7bfa7fc640 1 -- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bfc06bc00 msgr2=0x7f7bfc1448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.667+0000 7f7bfa7fc640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bfc06bc00 0x7f7bfc1448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.667+0000 7f7bfa7fc640 1 -- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bfc148590 msgr2=0x7f7bfc1472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:00.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.667+0000 7f7bfa7fc640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bfc148590 0x7f7bfc1472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:00.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.667+0000 7f7bfa7fc640 1 -- 172.21.15.87:0/3602247367 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec009b30 con 0x7f7bfc153920 2024-01-20T08:59:00.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.669+0000 7f7bfa7fc640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc143380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c500 tx=0x7f7bf000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:00.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.673+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0007500 con 0x7f7bfc153920 2024-01-20T08:59:00.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.673+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0007660 con 0x7f7bfc153920 2024-01-20T08:59:00.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.673+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00049e0 con 0x7f7bfc153920 2024-01-20T08:59:00.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.673+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc145d70 con 0x7f7bfc153920 2024-01-20T08:59:00.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.673+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc146330 con 0x7f7bfc153920 2024-01-20T08:59:00.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.674+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc06b9e0 con 0x7f7bfc153920 2024-01-20T08:59:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.915+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7bf0004b40 con 0x7f7bfc153920 2024-01-20T08:59:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.916+0000 7f7be3fff640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7be806af50 0x7f7be806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.916+0000 7f7bfaffd640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7be806af50 0x7f7be806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.916+0000 7f7bfaffd640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7be806af50 0x7f7be806d410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bec035000 tx=0x7f7bec030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.921+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7bf0013e20 con 0x7f7bfc153920 2024-01-20T08:59:00.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:00.932+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7bf0053f60 con 0x7f7bfc153920 2024-01-20T08:59:01.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.100+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bfc0c9700 con 0x7f7be806af50 2024-01-20T08:59:01.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.109+0000 7f7be3fff640 1 -- 172.21.15.87:0/3602247367 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371350 (secure 0 0 0) 0x7f7bec009510 con 0x7f7be806af50 2024-01-20T08:59:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:01.116 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7be806af50 msgr2=0x7f7be806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7be806af50 0x7f7be806d410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bec035000 tx=0x7f7bec030000 comp rx=0 tx=0).stop 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 msgr2=0x7f7bfc143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc143380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c500 tx=0x7f7bf000c9c0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 shutdown_connections 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7be806af50 0x7f7be806d410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bfc148590 0x7f7bfc1472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bfc153920 0x7f7bfc143380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 --2- 172.21.15.87:0/3602247367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bfc06bc00 0x7f7bfc1448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 >> 172.21.15.87:0/3602247367 conn(0x7f7bfc06a5b0 msgr2=0x7f7bfc1521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 shutdown_connections 2024-01-20T08:59:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.113+0000 7f7c019a9640 1 -- 172.21.15.87:0/3602247367 wait complete. 2024-01-20T08:59:01.125 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":866,"stamp":"2024-01-20T08:59:00.652384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39627569,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":1083,"num_read_kb":1128,"num_write":12670,"num_write_kb":354190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14976,"ondisk_log_size":14976,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461136,"kb_used_data":79888,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754513584,"statfs":{"total":773094113280,"available":772621910016,"internally_reserved":0,"allocated":81805312,"data_stored":80194922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390319664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61367,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003235"},"pg_stats":[{"pgid":"2.1f","version":"27'609","reported_seq":641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:34.122861+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:58:34.122861+0000","last_peered":"2024-01-20T08:58:34.122861+0000","last_clean":"2024-01-20T08:58:34.122861+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:58:34.122861+0000","last_undegraded":"2024-01-20T08:58:34.122861+0000","last_fullsized":"2024-01-20T08:58:34.122861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"27'27","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808166+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:58:53.808166+0000","last_peered":"2024-01-20T08:58:53.808166+0000","last_clean":"2024-01-20T08:58:53.808166+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:58:53.808166+0000","last_undegraded":"2024-01-20T08:58:53.808166+0000","last_fullsized":"2024-01-20T08:58:53.808166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"27'162","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.884124+0000","last_change":"2024-01-20T08:31:44.811554+0000","last_active":"2024-01-20T08:58:24.884124+0000","last_peered":"2024-01-20T08:58:24.884124+0000","last_clean":"2024-01-20T08:58:24.884124+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:58:24.884124+0000","last_undegraded":"2024-01-20T08:58:24.884124+0000","last_fullsized":"2024-01-20T08:58:24.884124+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:31:34.560783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":10773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'696","reported_seq":728,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.571964+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:58:47.571964+0000","last_peered":"2024-01-20T08:58:47.571964+0000","last_clean":"2024-01-20T08:58:47.571964+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:58:47.571964+0000","last_undegraded":"2024-01-20T08:58:47.571964+0000","last_fullsized":"2024-01-20T08:58:47.571964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"27'37","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843652+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:58:53.843652+0000","last_peered":"2024-01-20T08:58:53.843652+0000","last_clean":"2024-01-20T08:58:53.843652+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:58:53.843652+0000","last_undegraded":"2024-01-20T08:58:53.843652+0000","last_fullsized":"2024-01-20T08:58:53.843652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"27'136","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.633246+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:58:22.633246+0000","last_peered":"2024-01-20T08:58:22.633246+0000","last_clean":"2024-01-20T08:58:22.633246+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:58:22.633246+0000","last_undegraded":"2024-01-20T08:58:22.633246+0000","last_fullsized":"2024-01-20T08:58:22.633246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'1352","reported_seq":1382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.534987+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:58:46.534987+0000","last_peered":"2024-01-20T08:58:46.534987+0000","last_clean":"2024-01-20T08:58:46.534987+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:58:46.534987+0000","last_undegraded":"2024-01-20T08:58:46.534987+0000","last_fullsized":"2024-01-20T08:58:46.534987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.883892+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:58:53.883892+0000","last_peered":"2024-01-20T08:58:53.883892+0000","last_clean":"2024-01-20T08:58:53.883892+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:58:53.883892+0000","last_undegraded":"2024-01-20T08:58:53.883892+0000","last_fullsized":"2024-01-20T08:58:53.883892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"27'128","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:20.331609+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:58:20.331609+0000","last_peered":"2024-01-20T08:58:20.331609+0000","last_clean":"2024-01-20T08:58:20.331609+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:58:20.331609+0000","last_undegraded":"2024-01-20T08:58:20.331609+0000","last_fullsized":"2024-01-20T08:58:20.331609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:37.699537+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:58:37.699537+0000","last_peered":"2024-01-20T08:58:37.699537+0000","last_clean":"2024-01-20T08:58:37.699537+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:58:37.699537+0000","last_undegraded":"2024-01-20T08:58:37.699537+0000","last_fullsized":"2024-01-20T08:58:37.699537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"27'47","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878405+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:58:53.878405+0000","last_peered":"2024-01-20T08:58:53.878405+0000","last_clean":"2024-01-20T08:58:53.878405+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:58:53.878405+0000","last_undegraded":"2024-01-20T08:58:53.878405+0000","last_fullsized":"2024-01-20T08:58:53.878405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"27'149","reported_seq":175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.165676+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:58:23.165676+0000","last_peered":"2024-01-20T08:58:23.165676+0000","last_clean":"2024-01-20T08:58:23.165676+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:58:23.165676+0000","last_undegraded":"2024-01-20T08:58:23.165676+0000","last_fullsized":"2024-01-20T08:58:23.165676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":7763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.884346+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:58:53.884346+0000","last_peered":"2024-01-20T08:58:53.884346+0000","last_clean":"2024-01-20T08:58:53.884346+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:58:53.884346+0000","last_undegraded":"2024-01-20T08:58:53.884346+0000","last_fullsized":"2024-01-20T08:58:53.884346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'593","reported_seq":624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.320730+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:58:40.320730+0000","last_peered":"2024-01-20T08:58:40.320730+0000","last_clean":"2024-01-20T08:58:40.320730+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:58:40.320730+0000","last_undegraded":"2024-01-20T08:58:40.320730+0000","last_fullsized":"2024-01-20T08:58:40.320730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":4387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'109","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.837239+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:58:24.837239+0000","last_peered":"2024-01-20T08:58:24.837239+0000","last_clean":"2024-01-20T08:58:24.837239+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:58:24.837239+0000","last_undegraded":"2024-01-20T08:58:24.837239+0000","last_fullsized":"2024-01-20T08:58:24.837239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.840307+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:58:53.840307+0000","last_peered":"2024-01-20T08:58:53.840307+0000","last_clean":"2024-01-20T08:58:53.840307+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:58:53.840307+0000","last_undegraded":"2024-01-20T08:58:53.840307+0000","last_fullsized":"2024-01-20T08:58:53.840307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294867+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:58:39.294867+0000","last_peered":"2024-01-20T08:58:39.294867+0000","last_clean":"2024-01-20T08:58:39.294867+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:58:39.294867+0000","last_undegraded":"2024-01-20T08:58:39.294867+0000","last_fullsized":"2024-01-20T08:58:39.294867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"27'149","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.869606+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:58:23.869606+0000","last_peered":"2024-01-20T08:58:23.869606+0000","last_clean":"2024-01-20T08:58:23.869606+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:58:23.869606+0000","last_undegraded":"2024-01-20T08:58:23.869606+0000","last_fullsized":"2024-01-20T08:58:23.869606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.926690+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:58:53.926690+0000","last_peered":"2024-01-20T08:58:53.926690+0000","last_clean":"2024-01-20T08:58:53.926690+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:58:53.926690+0000","last_undegraded":"2024-01-20T08:58:53.926690+0000","last_fullsized":"2024-01-20T08:58:53.926690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'13","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.531366+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:58:46.531366+0000","last_peered":"2024-01-20T08:58:46.531366+0000","last_clean":"2024-01-20T08:58:46.531366+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:58:46.531366+0000","last_undegraded":"2024-01-20T08:58:46.531366+0000","last_fullsized":"2024-01-20T08:58:46.531366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"27'121","reported_seq":148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.284930+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:58:23.284930+0000","last_peered":"2024-01-20T08:58:23.284930+0000","last_clean":"2024-01-20T08:58:23.284930+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:58:23.284930+0000","last_undegraded":"2024-01-20T08:58:23.284930+0000","last_fullsized":"2024-01-20T08:58:23.284930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'32","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.847454+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:58:25.847454+0000","last_peered":"2024-01-20T08:58:25.847454+0000","last_clean":"2024-01-20T08:58:25.847454+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:58:25.847454+0000","last_undegraded":"2024-01-20T08:58:25.847454+0000","last_fullsized":"2024-01-20T08:58:25.847454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:42.498428+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:58:42.498428+0000","last_peered":"2024-01-20T08:58:42.498428+0000","last_clean":"2024-01-20T08:58:42.498428+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:58:42.498428+0000","last_undegraded":"2024-01-20T08:58:42.498428+0000","last_fullsized":"2024-01-20T08:58:42.498428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'112","reported_seq":139,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:21.538368+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:58:21.538368+0000","last_peered":"2024-01-20T08:58:21.538368+0000","last_clean":"2024-01-20T08:58:21.538368+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:58:21.538368+0000","last_undegraded":"2024-01-20T08:58:21.538368+0000","last_fullsized":"2024-01-20T08:58:21.538368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928171+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:58:53.928171+0000","last_peered":"2024-01-20T08:58:53.928171+0000","last_clean":"2024-01-20T08:58:53.928171+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:58:53.928171+0000","last_undegraded":"2024-01-20T08:58:53.928171+0000","last_fullsized":"2024-01-20T08:58:53.928171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"27'12","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:44.519111+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:58:44.519111+0000","last_peered":"2024-01-20T08:58:44.519111+0000","last_clean":"2024-01-20T08:58:44.519111+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:58:44.519111+0000","last_undegraded":"2024-01-20T08:58:44.519111+0000","last_fullsized":"2024-01-20T08:58:44.519111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.165734+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:58:23.165734+0000","last_peered":"2024-01-20T08:58:23.165734+0000","last_clean":"2024-01-20T08:58:23.165734+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:58:23.165734+0000","last_undegraded":"2024-01-20T08:58:23.165734+0000","last_fullsized":"2024-01-20T08:58:23.165734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929778+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:58:53.929778+0000","last_peered":"2024-01-20T08:58:53.929778+0000","last_clean":"2024-01-20T08:58:53.929778+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:58:53.929778+0000","last_undegraded":"2024-01-20T08:58:53.929778+0000","last_fullsized":"2024-01-20T08:58:53.929778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'315","reported_seq":346,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.274403+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:58:47.274403+0000","last_peered":"2024-01-20T08:58:47.274403+0000","last_clean":"2024-01-20T08:58:47.274403+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:58:47.274403+0000","last_undegraded":"2024-01-20T08:58:47.274403+0000","last_fullsized":"2024-01-20T08:58:47.274403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.953625+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:58:23.953625+0000","last_peered":"2024-01-20T08:58:23.953625+0000","last_clean":"2024-01-20T08:58:23.953625+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:58:23.953625+0000","last_undegraded":"2024-01-20T08:58:23.953625+0000","last_fullsized":"2024-01-20T08:58:23.953625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":11573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.800916+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:58:53.800916+0000","last_peered":"2024-01-20T08:58:53.800916+0000","last_clean":"2024-01-20T08:58:53.800916+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:58:53.800916+0000","last_undegraded":"2024-01-20T08:58:53.800916+0000","last_fullsized":"2024-01-20T08:58:53.800916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.323865+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:58:39.323865+0000","last_peered":"2024-01-20T08:58:39.323865+0000","last_clean":"2024-01-20T08:58:39.323865+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:58:39.323865+0000","last_undegraded":"2024-01-20T08:58:39.323865+0000","last_fullsized":"2024-01-20T08:58:39.323865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"27'126","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.957683+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:58:22.957683+0000","last_peered":"2024-01-20T08:58:22.957683+0000","last_clean":"2024-01-20T08:58:22.957683+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:58:22.957683+0000","last_undegraded":"2024-01-20T08:58:22.957683+0000","last_fullsized":"2024-01-20T08:58:22.957683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892100+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:58:53.892100+0000","last_peered":"2024-01-20T08:58:53.892100+0000","last_clean":"2024-01-20T08:58:53.892100+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:58:53.892100+0000","last_undegraded":"2024-01-20T08:58:53.892100+0000","last_fullsized":"2024-01-20T08:58:53.892100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:43.516881+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:58:43.516881+0000","last_peered":"2024-01-20T08:58:43.516881+0000","last_clean":"2024-01-20T08:58:43.516881+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:58:43.516881+0000","last_undegraded":"2024-01-20T08:58:43.516881+0000","last_fullsized":"2024-01-20T08:58:43.516881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"27'181","reported_seq":209,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.853812+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:58:22.853812+0000","last_peered":"2024-01-20T08:58:22.853812+0000","last_clean":"2024-01-20T08:58:22.853812+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:58:22.853812+0000","last_undegraded":"2024-01-20T08:58:22.853812+0000","last_fullsized":"2024-01-20T08:58:22.853812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019865+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:58:54.019865+0000","last_peered":"2024-01-20T08:58:54.019865+0000","last_clean":"2024-01-20T08:58:54.019865+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:58:54.019865+0000","last_undegraded":"2024-01-20T08:58:54.019865+0000","last_fullsized":"2024-01-20T08:58:54.019865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"27'13","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.315491+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:58:40.315491+0000","last_peered":"2024-01-20T08:58:40.315491+0000","last_clean":"2024-01-20T08:58:40.315491+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:58:40.315491+0000","last_undegraded":"2024-01-20T08:58:40.315491+0000","last_fullsized":"2024-01-20T08:58:40.315491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'113","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.935521+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:58:23.935521+0000","last_peered":"2024-01-20T08:58:23.935521+0000","last_clean":"2024-01-20T08:58:23.935521+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:58:23.935521+0000","last_undegraded":"2024-01-20T08:58:23.935521+0000","last_fullsized":"2024-01-20T08:58:23.935521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808962+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:58:53.808962+0000","last_peered":"2024-01-20T08:58:53.808962+0000","last_clean":"2024-01-20T08:58:53.808962+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:58:53.808962+0000","last_undegraded":"2024-01-20T08:58:53.808962+0000","last_fullsized":"2024-01-20T08:58:53.808962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"27'978","reported_seq":1977,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:49.545029+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:58:49.545029+0000","last_peered":"2024-01-20T08:58:49.545029+0000","last_clean":"2024-01-20T08:58:49.545029+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:58:49.545029+0000","last_undegraded":"2024-01-20T08:58:49.545029+0000","last_fullsized":"2024-01-20T08:58:49.545029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":978,"log_dups_size":0,"ondisk_log_size":978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210232,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":976,"num_write_kb":1034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"27'135","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.812343+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:58:25.812343+0000","last_peered":"2024-01-20T08:58:25.812343+0000","last_clean":"2024-01-20T08:58:25.812343+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:58:25.812343+0000","last_undegraded":"2024-01-20T08:58:25.812343+0000","last_fullsized":"2024-01-20T08:58:25.812343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118,"num_write_kb":10261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"27'28","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804624+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:58:53.804624+0000","last_peered":"2024-01-20T08:58:53.804624+0000","last_clean":"2024-01-20T08:58:53.804624+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:58:53.804624+0000","last_undegraded":"2024-01-20T08:58:53.804624+0000","last_fullsized":"2024-01-20T08:58:53.804624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"27'429","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.345853+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:58:41.345853+0000","last_peered":"2024-01-20T08:58:41.345853+0000","last_clean":"2024-01-20T08:58:41.345853+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:58:41.345853+0000","last_undegraded":"2024-01-20T08:58:41.345853+0000","last_fullsized":"2024-01-20T08:58:41.345853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":4309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"27'129","reported_seq":156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.864992+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:58:23.864992+0000","last_peered":"2024-01-20T08:58:23.864992+0000","last_clean":"2024-01-20T08:58:23.864992+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:58:23.864992+0000","last_undegraded":"2024-01-20T08:58:23.864992+0000","last_fullsized":"2024-01-20T08:58:23.864992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881046+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:58:53.881046+0000","last_peered":"2024-01-20T08:58:53.881046+0000","last_clean":"2024-01-20T08:58:53.881046+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:58:53.881046+0000","last_undegraded":"2024-01-20T08:58:53.881046+0000","last_fullsized":"2024-01-20T08:58:53.881046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"27'9","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.105404+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:58:33.105404+0000","last_peered":"2024-01-20T08:58:33.105404+0000","last_clean":"2024-01-20T08:58:33.105404+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:58:33.105404+0000","last_undegraded":"2024-01-20T08:58:33.105404+0000","last_fullsized":"2024-01-20T08:58:33.105404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:56:51.793427+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:56:51.793427+0000","last_peered":"2024-01-20T08:56:51.793427+0000","last_clean":"2024-01-20T08:56:51.793427+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:56:51.793427+0000","last_undegraded":"2024-01-20T08:56:51.793427+0000","last_fullsized":"2024-01-20T08:56:51.793427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'156","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.838096+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:58:24.838096+0000","last_peered":"2024-01-20T08:58:24.838096+0000","last_clean":"2024-01-20T08:58:24.838096+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:58:24.838096+0000","last_undegraded":"2024-01-20T08:58:24.838096+0000","last_fullsized":"2024-01-20T08:58:24.838096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":9711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.880987+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:58:53.880987+0000","last_peered":"2024-01-20T08:58:53.880987+0000","last_clean":"2024-01-20T08:58:53.880987+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:58:53.880987+0000","last_undegraded":"2024-01-20T08:58:53.880987+0000","last_fullsized":"2024-01-20T08:58:53.880987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569355+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:58:45.569355+0000","last_peered":"2024-01-20T08:58:45.569355+0000","last_clean":"2024-01-20T08:58:45.569355+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:58:45.569355+0000","last_undegraded":"2024-01-20T08:58:45.569355+0000","last_fullsized":"2024-01-20T08:58:45.569355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"27'130","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.565862+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:58:23.565862+0000","last_peered":"2024-01-20T08:58:23.565862+0000","last_clean":"2024-01-20T08:58:23.565862+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:58:23.565862+0000","last_undegraded":"2024-01-20T08:58:23.565862+0000","last_fullsized":"2024-01-20T08:58:23.565862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":4819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'39","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.021114+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:58:54.021114+0000","last_peered":"2024-01-20T08:58:54.021114+0000","last_clean":"2024-01-20T08:58:54.021114+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:58:54.021114+0000","last_undegraded":"2024-01-20T08:58:54.021114+0000","last_fullsized":"2024-01-20T08:58:54.021114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.352940+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:58:41.352940+0000","last_peered":"2024-01-20T08:58:41.352940+0000","last_clean":"2024-01-20T08:58:41.352940+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:58:41.352940+0000","last_undegraded":"2024-01-20T08:58:41.352940+0000","last_fullsized":"2024-01-20T08:58:41.352940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"27'188","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.886971+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:58:24.886971+0000","last_peered":"2024-01-20T08:58:24.886971+0000","last_clean":"2024-01-20T08:58:24.886971+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:58:24.886971+0000","last_undegraded":"2024-01-20T08:58:24.886971+0000","last_fullsized":"2024-01-20T08:58:24.886971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'35","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.845305+0000","last_change":"2024-01-20T08:31:44.212803+0000","last_active":"2024-01-20T08:58:53.845305+0000","last_peered":"2024-01-20T08:58:53.845305+0000","last_clean":"2024-01-20T08:58:53.845305+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:58:53.845305+0000","last_undegraded":"2024-01-20T08:58:53.845305+0000","last_fullsized":"2024-01-20T08:58:53.845305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:29.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.295236+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:58:39.295236+0000","last_peered":"2024-01-20T08:58:39.295236+0000","last_clean":"2024-01-20T08:58:39.295236+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:58:39.295236+0000","last_undegraded":"2024-01-20T08:58:39.295236+0000","last_fullsized":"2024-01-20T08:58:39.295236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"27'145","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.862293+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:58:24.862293+0000","last_peered":"2024-01-20T08:58:24.862293+0000","last_clean":"2024-01-20T08:58:24.862293+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:58:24.862293+0000","last_undegraded":"2024-01-20T08:58:24.862293+0000","last_fullsized":"2024-01-20T08:58:24.862293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.893905+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:58:53.893905+0000","last_peered":"2024-01-20T08:58:53.893905+0000","last_clean":"2024-01-20T08:58:53.893905+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:58:53.893905+0000","last_undegraded":"2024-01-20T08:58:53.893905+0000","last_fullsized":"2024-01-20T08:58:53.893905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:12.697455+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:58:12.697455+0000","last_peered":"2024-01-20T08:58:12.697455+0000","last_clean":"2024-01-20T08:58:12.697455+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:58:12.697455+0000","last_undegraded":"2024-01-20T08:58:12.697455+0000","last_fullsized":"2024-01-20T08:58:12.697455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"27'160","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.545680+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:58:23.545680+0000","last_peered":"2024-01-20T08:58:23.545680+0000","last_clean":"2024-01-20T08:58:23.545680+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:58:23.545680+0000","last_undegraded":"2024-01-20T08:58:23.545680+0000","last_fullsized":"2024-01-20T08:58:23.545680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":10942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"27'35","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.882452+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:58:53.882452+0000","last_peered":"2024-01-20T08:58:53.882452+0000","last_clean":"2024-01-20T08:58:53.882452+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:58:53.882452+0000","last_undegraded":"2024-01-20T08:58:53.882452+0000","last_fullsized":"2024-01-20T08:58:53.882452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:36.115658+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:58:36.115658+0000","last_peered":"2024-01-20T08:58:36.115658+0000","last_clean":"2024-01-20T08:58:36.115658+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:58:36.115658+0000","last_undegraded":"2024-01-20T08:58:36.115658+0000","last_fullsized":"2024-01-20T08:58:36.115658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"27'142","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.848076+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:58:24.848076+0000","last_peered":"2024-01-20T08:58:24.848076+0000","last_clean":"2024-01-20T08:58:24.848076+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:58:24.848076+0000","last_undegraded":"2024-01-20T08:58:24.848076+0000","last_fullsized":"2024-01-20T08:58:24.848076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881441+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:58:53.881441+0000","last_peered":"2024-01-20T08:58:53.881441+0000","last_clean":"2024-01-20T08:58:53.881441+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:58:53.881441+0000","last_undegraded":"2024-01-20T08:58:53.881441+0000","last_fullsized":"2024-01-20T08:58:53.881441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"27'11","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.570427+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:58:47.570427+0000","last_peered":"2024-01-20T08:58:47.570427+0000","last_clean":"2024-01-20T08:58:47.570427+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:58:47.570427+0000","last_undegraded":"2024-01-20T08:58:47.570427+0000","last_fullsized":"2024-01-20T08:58:47.570427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1169,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"27'133","reported_seq":159,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.549367+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:58:22.549367+0000","last_peered":"2024-01-20T08:58:22.549367+0000","last_clean":"2024-01-20T08:58:22.549367+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:58:22.549367+0000","last_undegraded":"2024-01-20T08:58:22.549367+0000","last_fullsized":"2024-01-20T08:58:22.549367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875228+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:58:53.875228+0000","last_peered":"2024-01-20T08:58:53.875228+0000","last_clean":"2024-01-20T08:58:53.875228+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:58:53.875228+0000","last_undegraded":"2024-01-20T08:58:53.875228+0000","last_fullsized":"2024-01-20T08:58:53.875228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.531270+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:58:45.531270+0000","last_peered":"2024-01-20T08:58:45.531270+0000","last_clean":"2024-01-20T08:58:45.531270+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:58:45.531270+0000","last_undegraded":"2024-01-20T08:58:45.531270+0000","last_fullsized":"2024-01-20T08:58:45.531270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"27'116","reported_seq":142,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.966157+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:58:23.966157+0000","last_peered":"2024-01-20T08:58:23.966157+0000","last_clean":"2024-01-20T08:58:23.966157+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:58:23.966157+0000","last_undegraded":"2024-01-20T08:58:23.966157+0000","last_fullsized":"2024-01-20T08:58:23.966157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"27'40","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.803393+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:58:53.803393+0000","last_peered":"2024-01-20T08:58:53.803393+0000","last_clean":"2024-01-20T08:58:53.803393+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:58:53.803393+0000","last_undegraded":"2024-01-20T08:58:53.803393+0000","last_fullsized":"2024-01-20T08:58:53.803393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"27'111","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.634588+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:58:53.634588+0000","last_peered":"2024-01-20T08:58:53.634588+0000","last_clean":"2024-01-20T08:58:53.634588+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:58:53.634588+0000","last_undegraded":"2024-01-20T08:58:53.634588+0000","last_fullsized":"2024-01-20T08:58:53.634588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069576,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"27'126","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.892559+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:58:23.892559+0000","last_peered":"2024-01-20T08:58:23.892559+0000","last_clean":"2024-01-20T08:58:23.892559+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:58:23.892559+0000","last_undegraded":"2024-01-20T08:58:23.892559+0000","last_fullsized":"2024-01-20T08:58:23.892559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.017745+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:58:54.017745+0000","last_peered":"2024-01-20T08:58:54.017745+0000","last_clean":"2024-01-20T08:58:54.017745+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:58:54.017745+0000","last_undegraded":"2024-01-20T08:58:54.017745+0000","last_fullsized":"2024-01-20T08:58:54.017745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'833","reported_seq":863,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:29.921887+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:58:29.921887+0000","last_peered":"2024-01-20T08:58:29.921887+0000","last_clean":"2024-01-20T08:58:29.921887+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:58:29.921887+0000","last_undegraded":"2024-01-20T08:58:29.921887+0000","last_fullsized":"2024-01-20T08:58:29.921887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"27'135","reported_seq":161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.466040+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:58:22.466040+0000","last_peered":"2024-01-20T08:58:22.466040+0000","last_clean":"2024-01-20T08:58:22.466040+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:58:22.466040+0000","last_undegraded":"2024-01-20T08:58:22.466040+0000","last_fullsized":"2024-01-20T08:58:22.466040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881221+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:58:53.881221+0000","last_peered":"2024-01-20T08:58:53.881221+0000","last_clean":"2024-01-20T08:58:53.881221+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:58:53.881221+0000","last_undegraded":"2024-01-20T08:58:53.881221+0000","last_fullsized":"2024-01-20T08:58:53.881221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"27'652","reported_seq":682,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.580377+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:58:47.580377+0000","last_peered":"2024-01-20T08:58:47.580377+0000","last_clean":"2024-01-20T08:58:47.580377+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:58:47.580377+0000","last_undegraded":"2024-01-20T08:58:47.580377+0000","last_fullsized":"2024-01-20T08:58:47.580377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.790975+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:58:24.790975+0000","last_peered":"2024-01-20T08:58:24.790975+0000","last_clean":"2024-01-20T08:58:24.790975+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:58:24.790975+0000","last_undegraded":"2024-01-20T08:58:24.790975+0000","last_fullsized":"2024-01-20T08:58:24.790975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":23255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.833806+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:58:53.833806+0000","last_peered":"2024-01-20T08:58:53.833806+0000","last_clean":"2024-01-20T08:58:53.833806+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:58:53.833806+0000","last_undegraded":"2024-01-20T08:58:53.833806+0000","last_fullsized":"2024-01-20T08:58:53.833806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.102700+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:58:33.102700+0000","last_peered":"2024-01-20T08:58:33.102700+0000","last_clean":"2024-01-20T08:58:33.102700+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:58:33.102700+0000","last_undegraded":"2024-01-20T08:58:33.102700+0000","last_fullsized":"2024-01-20T08:58:33.102700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"27'124","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.864733+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:58:24.864733+0000","last_peered":"2024-01-20T08:58:24.864733+0000","last_clean":"2024-01-20T08:58:24.864733+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:58:24.864733+0000","last_undegraded":"2024-01-20T08:58:24.864733+0000","last_fullsized":"2024-01-20T08:58:24.864733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895614+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:58:53.895614+0000","last_peered":"2024-01-20T08:58:53.895614+0000","last_clean":"2024-01-20T08:58:53.895614+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:58:53.895614+0000","last_undegraded":"2024-01-20T08:58:53.895614+0000","last_fullsized":"2024-01-20T08:58:53.895614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.632949+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:58:53.632949+0000","last_peered":"2024-01-20T08:58:53.632949+0000","last_clean":"2024-01-20T08:58:53.632949+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:58:53.632949+0000","last_undegraded":"2024-01-20T08:58:53.632949+0000","last_fullsized":"2024-01-20T08:58:53.632949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.554720+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:58:22.554720+0000","last_peered":"2024-01-20T08:58:22.554720+0000","last_clean":"2024-01-20T08:58:22.554720+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:58:22.554720+0000","last_undegraded":"2024-01-20T08:58:22.554720+0000","last_fullsized":"2024-01-20T08:58:22.554720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"27'130","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.958134+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:58:23.958134+0000","last_peered":"2024-01-20T08:58:23.958134+0000","last_clean":"2024-01-20T08:58:23.958134+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:58:23.958134+0000","last_undegraded":"2024-01-20T08:58:23.958134+0000","last_fullsized":"2024-01-20T08:58:23.958134+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":8002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.932256+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:58:53.932256+0000","last_peered":"2024-01-20T08:58:53.932256+0000","last_clean":"2024-01-20T08:58:53.932256+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:58:53.932256+0000","last_undegraded":"2024-01-20T08:58:53.932256+0000","last_fullsized":"2024-01-20T08:58:53.932256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.360632+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:58:41.360632+0000","last_peered":"2024-01-20T08:58:41.360632+0000","last_clean":"2024-01-20T08:58:41.360632+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:58:41.360632+0000","last_undegraded":"2024-01-20T08:58:41.360632+0000","last_fullsized":"2024-01-20T08:58:41.360632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"27'143","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.710016+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:58:24.710016+0000","last_peered":"2024-01-20T08:58:24.710016+0000","last_clean":"2024-01-20T08:58:24.710016+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:58:24.710016+0000","last_undegraded":"2024-01-20T08:58:24.710016+0000","last_fullsized":"2024-01-20T08:58:24.710016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":18955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'41","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.841394+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:58:53.841394+0000","last_peered":"2024-01-20T08:58:53.841394+0000","last_clean":"2024-01-20T08:58:53.841394+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:58:53.841394+0000","last_undegraded":"2024-01-20T08:58:53.841394+0000","last_fullsized":"2024-01-20T08:58:53.841394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294844+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:58:39.294844+0000","last_peered":"2024-01-20T08:58:39.294844+0000","last_clean":"2024-01-20T08:58:39.294844+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:58:39.294844+0000","last_undegraded":"2024-01-20T08:58:39.294844+0000","last_fullsized":"2024-01-20T08:58:39.294844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"27'675","reported_seq":705,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.711307+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:58:39.711307+0000","last_peered":"2024-01-20T08:58:39.711307+0000","last_clean":"2024-01-20T08:58:39.711307+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:58:39.711307+0000","last_undegraded":"2024-01-20T08:58:39.711307+0000","last_fullsized":"2024-01-20T08:58:39.711307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929004+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:58:53.929004+0000","last_peered":"2024-01-20T08:58:53.929004+0000","last_clean":"2024-01-20T08:58:53.929004+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:58:53.929004+0000","last_undegraded":"2024-01-20T08:58:53.929004+0000","last_fullsized":"2024-01-20T08:58:53.929004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"27'89","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.877290+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:58:23.877290+0000","last_peered":"2024-01-20T08:58:23.877290+0000","last_clean":"2024-01-20T08:58:23.877290+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:58:23.877290+0000","last_undegraded":"2024-01-20T08:58:23.877290+0000","last_fullsized":"2024-01-20T08:58:23.877290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":9333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'689","reported_seq":719,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569769+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:58:45.569769+0000","last_peered":"2024-01-20T08:58:45.569769+0000","last_clean":"2024-01-20T08:58:45.569769+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:58:45.569769+0000","last_undegraded":"2024-01-20T08:58:45.569769+0000","last_fullsized":"2024-01-20T08:58:45.569769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"27'33","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888587+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:58:53.888587+0000","last_peered":"2024-01-20T08:58:53.888587+0000","last_clean":"2024-01-20T08:58:53.888587+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:58:53.888587+0000","last_undegraded":"2024-01-20T08:58:53.888587+0000","last_fullsized":"2024-01-20T08:58:53.888587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"27'147","reported_seq":173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.840471+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:58:24.840471+0000","last_peered":"2024-01-20T08:58:24.840471+0000","last_clean":"2024-01-20T08:58:24.840471+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:58:24.840471+0000","last_undegraded":"2024-01-20T08:58:24.840471+0000","last_fullsized":"2024-01-20T08:58:24.840471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":10430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892176+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:58:53.892176+0000","last_peered":"2024-01-20T08:58:53.892176+0000","last_clean":"2024-01-20T08:58:53.892176+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:58:53.892176+0000","last_undegraded":"2024-01-20T08:58:53.892176+0000","last_fullsized":"2024-01-20T08:58:53.892176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843335+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:58:53.843335+0000","last_peered":"2024-01-20T08:58:53.843335+0000","last_clean":"2024-01-20T08:58:53.843335+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:58:53.843335+0000","last_undegraded":"2024-01-20T08:58:53.843335+0000","last_fullsized":"2024-01-20T08:58:53.843335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"27'44","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875423+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:58:53.875423+0000","last_peered":"2024-01-20T08:58:53.875423+0000","last_clean":"2024-01-20T08:58:53.875423+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:58:53.875423+0000","last_undegraded":"2024-01-20T08:58:53.875423+0000","last_fullsized":"2024-01-20T08:58:53.875423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.839038+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:58:53.839038+0000","last_peered":"2024-01-20T08:58:53.839038+0000","last_clean":"2024-01-20T08:58:53.839038+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:58:53.839038+0000","last_undegraded":"2024-01-20T08:58:53.839038+0000","last_fullsized":"2024-01-20T08:58:53.839038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"27'27","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.807055+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:58:53.807055+0000","last_peered":"2024-01-20T08:58:53.807055+0000","last_clean":"2024-01-20T08:58:53.807055+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:58:53.807055+0000","last_undegraded":"2024-01-20T08:58:53.807055+0000","last_fullsized":"2024-01-20T08:58:53.807055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'31","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.802128+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:58:53.802128+0000","last_peered":"2024-01-20T08:58:53.802128+0000","last_clean":"2024-01-20T08:58:53.802128+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:58:53.802128+0000","last_undegraded":"2024-01-20T08:58:53.802128+0000","last_fullsized":"2024-01-20T08:58:53.802128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"27'44","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895227+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:58:53.895227+0000","last_peered":"2024-01-20T08:58:53.895227+0000","last_clean":"2024-01-20T08:58:53.895227+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:58:53.895227+0000","last_undegraded":"2024-01-20T08:58:53.895227+0000","last_fullsized":"2024-01-20T08:58:53.895227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808904+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:58:53.808904+0000","last_peered":"2024-01-20T08:58:53.808904+0000","last_clean":"2024-01-20T08:58:53.808904+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:58:53.808904+0000","last_undegraded":"2024-01-20T08:58:53.808904+0000","last_fullsized":"2024-01-20T08:58:53.808904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.871511+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:58:53.871511+0000","last_peered":"2024-01-20T08:58:53.871511+0000","last_clean":"2024-01-20T08:58:53.871511+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:58:53.871511+0000","last_undegraded":"2024-01-20T08:58:53.871511+0000","last_fullsized":"2024-01-20T08:58:53.871511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"27'42","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.837960+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:58:53.837960+0000","last_peered":"2024-01-20T08:58:53.837960+0000","last_clean":"2024-01-20T08:58:53.837960+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:58:53.837960+0000","last_undegraded":"2024-01-20T08:58:53.837960+0000","last_fullsized":"2024-01-20T08:58:53.837960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"27'38","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888654+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:58:53.888654+0000","last_peered":"2024-01-20T08:58:53.888654+0000","last_clean":"2024-01-20T08:58:53.888654+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:58:53.888654+0000","last_undegraded":"2024-01-20T08:58:53.888654+0000","last_fullsized":"2024-01-20T08:58:53.888654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'47","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928357+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:58:53.928357+0000","last_peered":"2024-01-20T08:58:53.928357+0000","last_clean":"2024-01-20T08:58:53.928357+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:58:53.928357+0000","last_undegraded":"2024-01-20T08:58:53.928357+0000","last_fullsized":"2024-01-20T08:58:53.928357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.021424+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:58:54.021424+0000","last_peered":"2024-01-20T08:58:54.021424+0000","last_clean":"2024-01-20T08:58:54.021424+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:58:54.021424+0000","last_undegraded":"2024-01-20T08:58:54.021424+0000","last_fullsized":"2024-01-20T08:58:54.021424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878349+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:58:53.878349+0000","last_peered":"2024-01-20T08:58:53.878349+0000","last_clean":"2024-01-20T08:58:53.878349+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:58:53.878349+0000","last_undegraded":"2024-01-20T08:58:53.878349+0000","last_fullsized":"2024-01-20T08:58:53.878349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"27'45","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.846302+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:58:53.846302+0000","last_peered":"2024-01-20T08:58:53.846302+0000","last_clean":"2024-01-20T08:58:53.846302+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:58:53.846302+0000","last_undegraded":"2024-01-20T08:58:53.846302+0000","last_fullsized":"2024-01-20T08:58:53.846302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"27'38","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.870937+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:58:53.870937+0000","last_peered":"2024-01-20T08:58:53.870937+0000","last_clean":"2024-01-20T08:58:53.870937+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:58:53.870937+0000","last_undegraded":"2024-01-20T08:58:53.870937+0000","last_fullsized":"2024-01-20T08:58:53.870937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881740+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:58:53.881740+0000","last_peered":"2024-01-20T08:58:53.881740+0000","last_clean":"2024-01-20T08:58:53.881740+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:58:53.881740+0000","last_undegraded":"2024-01-20T08:58:53.881740+0000","last_fullsized":"2024-01-20T08:58:53.881740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.834894+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:58:53.834894+0000","last_peered":"2024-01-20T08:58:53.834894+0000","last_clean":"2024-01-20T08:58:53.834894+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:58:53.834894+0000","last_undegraded":"2024-01-20T08:58:53.834894+0000","last_fullsized":"2024-01-20T08:58:53.834894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804360+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:58:53.804360+0000","last_peered":"2024-01-20T08:58:53.804360+0000","last_clean":"2024-01-20T08:58:53.804360+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:58:53.804360+0000","last_undegraded":"2024-01-20T08:58:53.804360+0000","last_fullsized":"2024-01-20T08:58:53.804360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806862+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:58:53.806862+0000","last_peered":"2024-01-20T08:58:53.806862+0000","last_clean":"2024-01-20T08:58:53.806862+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:58:53.806862+0000","last_undegraded":"2024-01-20T08:58:53.806862+0000","last_fullsized":"2024-01-20T08:58:53.806862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023455+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:58:54.023455+0000","last_peered":"2024-01-20T08:58:54.023455+0000","last_clean":"2024-01-20T08:58:54.023455+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:58:54.023455+0000","last_undegraded":"2024-01-20T08:58:54.023455+0000","last_fullsized":"2024-01-20T08:58:54.023455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022954+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:58:54.022954+0000","last_peered":"2024-01-20T08:58:54.022954+0000","last_clean":"2024-01-20T08:58:54.022954+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:58:54.022954+0000","last_undegraded":"2024-01-20T08:58:54.022954+0000","last_fullsized":"2024-01-20T08:58:54.022954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023780+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:58:54.023780+0000","last_peered":"2024-01-20T08:58:54.023780+0000","last_clean":"2024-01-20T08:58:54.023780+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:58:54.023780+0000","last_undegraded":"2024-01-20T08:58:54.023780+0000","last_fullsized":"2024-01-20T08:58:54.023780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.889696+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:58:53.889696+0000","last_peered":"2024-01-20T08:58:53.889696+0000","last_clean":"2024-01-20T08:58:53.889696+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:58:53.889696+0000","last_undegraded":"2024-01-20T08:58:53.889696+0000","last_fullsized":"2024-01-20T08:58:53.889696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843441+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:58:53.843441+0000","last_peered":"2024-01-20T08:58:53.843441+0000","last_clean":"2024-01-20T08:58:53.843441+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:58:53.843441+0000","last_undegraded":"2024-01-20T08:58:53.843441+0000","last_fullsized":"2024-01-20T08:58:53.843441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.017192+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:58:54.017192+0000","last_peered":"2024-01-20T08:58:54.017192+0000","last_clean":"2024-01-20T08:58:54.017192+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:58:54.017192+0000","last_undegraded":"2024-01-20T08:58:54.017192+0000","last_fullsized":"2024-01-20T08:58:54.017192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892312+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:58:53.892312+0000","last_peered":"2024-01-20T08:58:53.892312+0000","last_clean":"2024-01-20T08:58:53.892312+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:58:53.892312+0000","last_undegraded":"2024-01-20T08:58:53.892312+0000","last_fullsized":"2024-01-20T08:58:53.892312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"27'30","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019038+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:58:54.019038+0000","last_peered":"2024-01-20T08:58:54.019038+0000","last_clean":"2024-01-20T08:58:54.019038+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:58:54.019038+0000","last_undegraded":"2024-01-20T08:58:54.019038+0000","last_fullsized":"2024-01-20T08:58:54.019038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'25","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892157+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:58:53.892157+0000","last_peered":"2024-01-20T08:58:53.892157+0000","last_clean":"2024-01-20T08:58:53.892157+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:58:53.892157+0000","last_undegraded":"2024-01-20T08:58:53.892157+0000","last_fullsized":"2024-01-20T08:58:53.892157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'41","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.842487+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:58:53.842487+0000","last_peered":"2024-01-20T08:58:53.842487+0000","last_clean":"2024-01-20T08:58:53.842487+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:58:53.842487+0000","last_undegraded":"2024-01-20T08:58:53.842487+0000","last_fullsized":"2024-01-20T08:58:53.842487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806164+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:58:53.806164+0000","last_peered":"2024-01-20T08:58:53.806164+0000","last_clean":"2024-01-20T08:58:53.806164+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:58:53.806164+0000","last_undegraded":"2024-01-20T08:58:53.806164+0000","last_fullsized":"2024-01-20T08:58:53.806164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"27'26","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022116+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:58:54.022116+0000","last_peered":"2024-01-20T08:58:54.022116+0000","last_clean":"2024-01-20T08:58:54.022116+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:58:54.022116+0000","last_undegraded":"2024-01-20T08:58:54.022116+0000","last_fullsized":"2024-01-20T08:58:54.022116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":308552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4420,"ondisk_log_size":4420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39037201,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":8593,"num_write_kb":43222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78323712,"data_stored":78074402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8116,"ondisk_log_size":8116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509784,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39732,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332108,"statfs":{"total":96636764160,"available":96596078592,"internally_reserved":0,"allocated":4501504,"data_stored":4315319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46999999999999997,"15min":0.45300000000000001},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.23899999999999999},"max":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.25800000000000001},"max":{"1min":0.60999999999999999,"5min":0.66200000000000003,"15min":0.81999999999999995},"last":0.46999999999999997}]},{"osd":1,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.67900000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28999999999999998},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.67500000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.70899999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.628,"5min":0.68500000000000005,"15min":0.72699999999999998},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.30299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.74199999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44800000000000001,"15min":0.45400000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.23799999999999999},"max":{"1min":0.63500000000000001,"5min":0.67400000000000004,"15min":0.76700000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75800000000000001,"15min":0.876},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.81000000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.79400000000000004},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.27600000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.90900000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"last":0.54400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424509784,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67036,"kb_used_data":13652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94304804,"statfs":{"total":96636764160,"available":96568119296,"internally_reserved":0,"allocated":13979648,"data_stored":13773597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.29099999999999998},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17199999999999999},"max":{"1min":0.41899999999999998,"5min":0.441,"15min":0.72599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.19500000000000001,"15min":0.19400000000000001},"max":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.58799999999999997},"last":0.28999999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.503,"15min":0.61799999999999999},"last":0.315},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.216,"15min":0.21099999999999999},"max":{"1min":0.39500000000000002,"5min":0.51000000000000001,"15min":0.79600000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.69499999999999995},"last":0.37}]},{"osd":4,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.72599999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.77600000000000002,"15min":0.91400000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.48499999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.78500000000000003},"last":0.497}]}]},{"osd":5,"up_from":14,"seq":60129542486,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70924,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300916,"statfs":{"total":96636764160,"available":96564137984,"internally_reserved":0,"allocated":13701120,"data_stored":13502280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"max":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.69599999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72099999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.66000000000000003},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.72899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.76700000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.80900000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44,"15min":0.439},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48399999999999999,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.26500000000000001},"max":{"1min":0.56799999999999995,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.81599999999999995},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.80300000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59399999999999997,"5min":0.76200000000000001,"15min":0.85799999999999998},"last":0.44400000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542485,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66068,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305772,"statfs":{"total":96636764160,"available":96569110528,"internally_reserved":0,"allocated":8728576,"data_stored":8508591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.61299999999999999,"15min":0.70199999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.629,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.31}]},{"osd":2,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.31,"15min":0.26500000000000001},"max":{"1min":0.51400000000000001,"5min":0.69599999999999995,"15min":0.73299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.315,"5min":0.31,"15min":0.28599999999999998},"max":{"1min":0.54700000000000004,"5min":0.622,"15min":0.66700000000000004},"last":0.38800000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57799999999999996,"5min":0.71899999999999997,"15min":0.73999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.254},"max":{"1min":0.61899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.26000000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.59099999999999997,"5min":0.75700000000000001,"15min":0.78400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":0.75600000000000001},"last":0.27200000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66000000000000003,"5min":0.78300000000000003,"15min":0.79100000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":0.71699999999999997,"15min":0.79000000000000004},"last":0.28899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542486,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43824,"kb_used_data":8488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328016,"statfs":{"total":96636764160,"available":96591888384,"internally_reserved":0,"allocated":8691712,"data_stored":8506821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.45800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29999999999999999,"15min":0.307},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":0.38100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.26300000000000001}]},{"osd":1,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.502,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.308}]},{"osd":3,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33000000000000002,"15min":0.33200000000000002},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.378,"5min":0.55900000000000005,"15min":0.60799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34599999999999997}]},{"osd":4,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.71399999999999997,"15min":0.82099999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.376}]},{"osd":5,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.86699999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.80500000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.69999999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.76000000000000001,"15min":0.79300000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.36399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85299999999999998},"last":0.439}]}]},{"osd":1,"up_from":14,"seq":60129542487,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9289728,"data_stored":9097863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30099999999999999,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.55700000000000005,"15min":0.59499999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.32000000000000001,"15min":0.31},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20000000000000001},"max":{"1min":0.40600000000000003,"5min":0.45500000000000002,"15min":0.54400000000000004},"last":0.307}]},{"osd":2,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.308,"15min":0.314},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.35399999999999998,"5min":0.47999999999999998,"15min":0.49199999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":0.45800000000000002,"5min":0.624,"15min":0.624},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.59399999999999997,"15min":0.63},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.44700000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.27900000000000003},"max":{"1min":0.80700000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39300000000000002}]},{"osd":5,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.876},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":0.873,"15min":0.873},"last":0.49199999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46400000000000002}]}]},{"osd":7,"up_from":13,"seq":55834575190,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63212,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308628,"statfs":{"total":96636764160,"available":96572035072,"internally_reserved":0,"allocated":5804032,"data_stored":5592459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.68999999999999995},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.54200000000000004,"5min":0.65100000000000002,"15min":0.73499999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.37,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.78000000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23000000000000001},"max":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.75800000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26600000000000001},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.79900000000000004},"last":0.40400000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.76400000000000001,"15min":0.77800000000000002},"last":0.443}]},{"osd":6,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.371},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.52200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575190,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61772,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310068,"statfs":{"total":96636764160,"available":96573509632,"internally_reserved":0,"allocated":17108992,"data_stored":16897992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.505,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.31,"15min":0.30399999999999999},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.41699999999999998,"5min":0.47599999999999998,"15min":0.51200000000000001},"last":0.30399999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47199999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.42299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29399999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44700000000000001,"5min":0.625,"15min":0.625},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.373,"15min":0.371},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.434,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.78500000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.71399999999999997,"5min":0.80000000000000004,"15min":0.85499999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.76100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.34000000000000002},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.497},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13692928,"data_stored":13656124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8409088,"data_stored":8390022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":8389348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16822272,"data_stored":16780519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8441856,"data_stored":8391118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12820480,"data_stored":12794439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4214784,"data_stored":4197846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":5474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:59:01.130 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-20T08:59:01.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:59:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.266+0000 7f37e29c5640 1 Processor -- start 2024-01-20T08:59:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.266+0000 7f37e29c5640 1 -- start start 2024-01-20T08:59:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.266+0000 7f37e29c5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc1501c0 0x7f37dc0c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.266+0000 7f37e29c5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f37dc0c8380 0x7f37dc159610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.266+0000 7f37e29c5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37dc06f800 0x7f37dc159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.266+0000 7f37e29c5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc0c2710 con 0x7f37dc0c8380 2024-01-20T08:59:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37e29c5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc06fd40 con 0x7f37dc06f800 2024-01-20T08:59:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37e29c5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc0c3030 con 0x7f37dc1501c0 2024-01-20T08:59:01.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37e0f62640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37dc06f800 0x7f37dc159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37db7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f37dc0c8380 0x7f37dc159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc1501c0 0x7f37dc0c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37e0f62640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37dc06f800 0x7f37dc159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58074/0 (socket says 172.21.15.87:58074) 2024-01-20T08:59:01.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.267+0000 7f37e0f62640 1 -- 172.21.15.87:0/3007370489 learned_addr learned my addr 172.21.15.87:0/3007370489 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.270+0000 7f37e0f62640 1 -- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc1501c0 msgr2=0x7f37dc0c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.270+0000 7f37e0f62640 1 --2- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc1501c0 0x7f37dc0c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.270+0000 7f37e0f62640 1 -- 172.21.15.87:0/3007370489 >> v2:172.21.15.87:3300/0 conn(0x7f37dc0c8380 msgr2=0x7f37dc159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.270+0000 7f37e0f62640 1 --2- 172.21.15.87:0/3007370489 >> v2:172.21.15.87:3300/0 conn(0x7f37dc0c8380 0x7f37dc159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.270+0000 7f37e0f62640 1 -- 172.21.15.87:0/3007370489 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc070000 con 0x7f37dc06f800 2024-01-20T08:59:01.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.271+0000 7f37e0f62640 1 --2- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc06f800 0x7f37dc159b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37dc157150 tx=0x7f37d402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c90b04b5c09b9bac server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:01.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.275+0000 7f37daffd640 1 -- 172.21.15.87:0/3007370489 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d402c9e0 con 0x7f37dc06f800 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.275+0000 7f37daffd640 1 -- 172.21.15.87:0/3007370489 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d402cb40 con 0x7f37dc06f800 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.275+0000 7f37daffd640 1 -- 172.21.15.87:0/3007370489 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40343f0 con 0x7f37dc06f800 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.275+0000 7f37e29c5640 1 -- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc06f800 msgr2=0x7f37dc159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.275+0000 7f37e29c5640 1 --2- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc06f800 0x7f37dc159b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37dc157150 tx=0x7f37d402be40 comp rx=0 tx=0).stop 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 -- 172.21.15.87:0/3007370489 shutdown_connections 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 --2- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc06f800 0x7f37dc159b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 --2- 172.21.15.87:0/3007370489 >> v2:172.21.15.87:3300/0 conn(0x7f37dc0c8380 0x7f37dc159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 --2- 172.21.15.87:0/3007370489 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc1501c0 0x7f37dc0c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 -- 172.21.15.87:0/3007370489 >> 172.21.15.87:0/3007370489 conn(0x7f37dc0cbe60 msgr2=0x7f37dc0cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 -- 172.21.15.87:0/3007370489 shutdown_connections 2024-01-20T08:59:01.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.276+0000 7f37e29c5640 1 -- 172.21.15.87:0/3007370489 wait complete. 2024-01-20T08:59:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.277+0000 7f37e29c5640 1 Processor -- start 2024-01-20T08:59:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.277+0000 7f37e29c5640 1 -- start start 2024-01-20T08:59:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e29c5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc06f800 0x7f37dc148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e29c5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 0x7f37dc15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e29c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37dc1501c0 0x7f37dc15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e29c5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc15c300 con 0x7f37dc1501c0 2024-01-20T08:59:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e29c5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc15ba60 con 0x7f37dc0c8380 2024-01-20T08:59:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e29c5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc0c4ca0 con 0x7f37dc06f800 2024-01-20T08:59:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37e0f62640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37dc1501c0 0x7f37dc15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 0x7f37dc15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.278+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc06f800 0x7f37dc148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.279+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 0x7f37dc15ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58078/0 (socket says 172.21.15.87:58078) 2024-01-20T08:59:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.279+0000 7f37e0f62640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37dc1501c0 0x7f37dc15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60020/0 (socket says 172.21.15.87:60020) 2024-01-20T08:59:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.279+0000 7f37db7fe640 1 -- 172.21.15.87:0/1695371750 learned_addr learned my addr 172.21.15.87:0/1695371750 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.282+0000 7f37db7fe640 1 -- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc06f800 msgr2=0x7f37dc148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.282+0000 7f37db7fe640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc06f800 0x7f37dc148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.282+0000 7f37db7fe640 1 -- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37dc1501c0 msgr2=0x7f37dc15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.282+0000 7f37db7fe640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37dc1501c0 0x7f37dc15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.282+0000 7f37db7fe640 1 -- 172.21.15.87:0/1695371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d4009b30 con 0x7f37dc0c8380 2024-01-20T08:59:01.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.284+0000 7f37db7fe640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 0x7f37dc15ccf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37d000e860 tx=0x7f37d000ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:01.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.286+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00095f0 con 0x7f37dc0c8380 2024-01-20T08:59:01.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.286+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc0c5e40 con 0x7f37dc0c8380 2024-01-20T08:59:01.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.286+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37dc15b1e0 con 0x7f37dc0c8380 2024-01-20T08:59:01.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.287+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37dc15b500 con 0x7f37dc0c8380 2024-01-20T08:59:01.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.288+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0005990 con 0x7f37dc0c8380 2024-01-20T08:59:01.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.288+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0005d00 con 0x7f37dc0c8380 2024-01-20T08:59:01.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.530+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f37d000c940 con 0x7f37dc0c8380 2024-01-20T08:59:01.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.530+0000 7f37d8ff9640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f37c806af00 0x7f37c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.530+0000 7f37dbfff640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f37c806af00 0x7f37c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.531+0000 7f37dbfff640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f37c806af00 0x7f37c806d3c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37dc15a280 tx=0x7f37cc009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:01.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.532+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f37d0058a40 con 0x7f37dc0c8380 2024-01-20T08:59:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.543+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f37d0054d80 con 0x7f37dc0c8380 2024-01-20T08:59:01.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.725+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37dc0c1f10 con 0x7f37c806af00 2024-01-20T08:59:01.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.734+0000 7f37d8ff9640 1 -- 172.21.15.87:0/1695371750 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371350 (secure 0 0 0) 0x7f37cc011a30 con 0x7f37c806af00 2024-01-20T08:59:01.739 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:01.741 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f37c806af00 msgr2=0x7f37c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f37c806af00 0x7f37c806d3c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37dc15a280 tx=0x7f37cc009190 comp rx=0 tx=0).stop 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 msgr2=0x7f37dc15ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 0x7f37dc15ccf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37d000e860 tx=0x7f37d000ed20 comp rx=0 tx=0).stop 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 shutdown_connections 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f37c806af00 0x7f37c806d3c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37dc1501c0 0x7f37dc15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37dc0c8380 0x7f37dc15ccf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 --2- 172.21.15.87:0/1695371750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37dc06f800 0x7f37dc148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 >> 172.21.15.87:0/1695371750 conn(0x7f37dc0cbe60 msgr2=0x7f37dc14e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:01.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 shutdown_connections 2024-01-20T08:59:01.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.738+0000 7f37e29c5640 1 -- 172.21.15.87:0/1695371750 wait complete. 2024-01-20T08:59:01.752 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":866,"stamp":"2024-01-20T08:59:00.652384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39627569,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":1083,"num_read_kb":1128,"num_write":12670,"num_write_kb":354190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14976,"ondisk_log_size":14976,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461136,"kb_used_data":79888,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754513584,"statfs":{"total":773094113280,"available":772621910016,"internally_reserved":0,"allocated":81805312,"data_stored":80194922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390319664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61367,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003235"},"pg_stats":[{"pgid":"2.1f","version":"27'609","reported_seq":641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:34.122861+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:58:34.122861+0000","last_peered":"2024-01-20T08:58:34.122861+0000","last_clean":"2024-01-20T08:58:34.122861+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:58:34.122861+0000","last_undegraded":"2024-01-20T08:58:34.122861+0000","last_fullsized":"2024-01-20T08:58:34.122861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"27'27","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808166+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:58:53.808166+0000","last_peered":"2024-01-20T08:58:53.808166+0000","last_clean":"2024-01-20T08:58:53.808166+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:58:53.808166+0000","last_undegraded":"2024-01-20T08:58:53.808166+0000","last_fullsized":"2024-01-20T08:58:53.808166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"27'162","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.884124+0000","last_change":"2024-01-20T08:31:44.811554+0000","last_active":"2024-01-20T08:58:24.884124+0000","last_peered":"2024-01-20T08:58:24.884124+0000","last_clean":"2024-01-20T08:58:24.884124+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:58:24.884124+0000","last_undegraded":"2024-01-20T08:58:24.884124+0000","last_fullsized":"2024-01-20T08:58:24.884124+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:31:34.560783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":10773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'696","reported_seq":728,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.571964+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:58:47.571964+0000","last_peered":"2024-01-20T08:58:47.571964+0000","last_clean":"2024-01-20T08:58:47.571964+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:58:47.571964+0000","last_undegraded":"2024-01-20T08:58:47.571964+0000","last_fullsized":"2024-01-20T08:58:47.571964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"27'37","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843652+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:58:53.843652+0000","last_peered":"2024-01-20T08:58:53.843652+0000","last_clean":"2024-01-20T08:58:53.843652+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:58:53.843652+0000","last_undegraded":"2024-01-20T08:58:53.843652+0000","last_fullsized":"2024-01-20T08:58:53.843652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"27'136","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.633246+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:58:22.633246+0000","last_peered":"2024-01-20T08:58:22.633246+0000","last_clean":"2024-01-20T08:58:22.633246+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:58:22.633246+0000","last_undegraded":"2024-01-20T08:58:22.633246+0000","last_fullsized":"2024-01-20T08:58:22.633246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:45.359455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'1352","reported_seq":1382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.534987+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:58:46.534987+0000","last_peered":"2024-01-20T08:58:46.534987+0000","last_clean":"2024-01-20T08:58:46.534987+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:58:46.534987+0000","last_undegraded":"2024-01-20T08:58:46.534987+0000","last_fullsized":"2024-01-20T08:58:46.534987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.883892+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:58:53.883892+0000","last_peered":"2024-01-20T08:58:53.883892+0000","last_clean":"2024-01-20T08:58:53.883892+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:58:53.883892+0000","last_undegraded":"2024-01-20T08:58:53.883892+0000","last_fullsized":"2024-01-20T08:58:53.883892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"27'128","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:20.331609+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:58:20.331609+0000","last_peered":"2024-01-20T08:58:20.331609+0000","last_clean":"2024-01-20T08:58:20.331609+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:58:20.331609+0000","last_undegraded":"2024-01-20T08:58:20.331609+0000","last_fullsized":"2024-01-20T08:58:20.331609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:37.699537+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:58:37.699537+0000","last_peered":"2024-01-20T08:58:37.699537+0000","last_clean":"2024-01-20T08:58:37.699537+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:58:37.699537+0000","last_undegraded":"2024-01-20T08:58:37.699537+0000","last_fullsized":"2024-01-20T08:58:37.699537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"27'47","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878405+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:58:53.878405+0000","last_peered":"2024-01-20T08:58:53.878405+0000","last_clean":"2024-01-20T08:58:53.878405+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:58:53.878405+0000","last_undegraded":"2024-01-20T08:58:53.878405+0000","last_fullsized":"2024-01-20T08:58:53.878405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"27'149","reported_seq":175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.165676+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:58:23.165676+0000","last_peered":"2024-01-20T08:58:23.165676+0000","last_clean":"2024-01-20T08:58:23.165676+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:58:23.165676+0000","last_undegraded":"2024-01-20T08:58:23.165676+0000","last_fullsized":"2024-01-20T08:58:23.165676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:08.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":7763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.884346+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:58:53.884346+0000","last_peered":"2024-01-20T08:58:53.884346+0000","last_clean":"2024-01-20T08:58:53.884346+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:58:53.884346+0000","last_undegraded":"2024-01-20T08:58:53.884346+0000","last_fullsized":"2024-01-20T08:58:53.884346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'593","reported_seq":624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.320730+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:58:40.320730+0000","last_peered":"2024-01-20T08:58:40.320730+0000","last_clean":"2024-01-20T08:58:40.320730+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:58:40.320730+0000","last_undegraded":"2024-01-20T08:58:40.320730+0000","last_fullsized":"2024-01-20T08:58:40.320730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":4387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'109","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.837239+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:58:24.837239+0000","last_peered":"2024-01-20T08:58:24.837239+0000","last_clean":"2024-01-20T08:58:24.837239+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:58:24.837239+0000","last_undegraded":"2024-01-20T08:58:24.837239+0000","last_fullsized":"2024-01-20T08:58:24.837239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.840307+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:58:53.840307+0000","last_peered":"2024-01-20T08:58:53.840307+0000","last_clean":"2024-01-20T08:58:53.840307+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:58:53.840307+0000","last_undegraded":"2024-01-20T08:58:53.840307+0000","last_fullsized":"2024-01-20T08:58:53.840307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294867+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:58:39.294867+0000","last_peered":"2024-01-20T08:58:39.294867+0000","last_clean":"2024-01-20T08:58:39.294867+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:58:39.294867+0000","last_undegraded":"2024-01-20T08:58:39.294867+0000","last_fullsized":"2024-01-20T08:58:39.294867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"27'149","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.869606+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:58:23.869606+0000","last_peered":"2024-01-20T08:58:23.869606+0000","last_clean":"2024-01-20T08:58:23.869606+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:58:23.869606+0000","last_undegraded":"2024-01-20T08:58:23.869606+0000","last_fullsized":"2024-01-20T08:58:23.869606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.926690+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:58:53.926690+0000","last_peered":"2024-01-20T08:58:53.926690+0000","last_clean":"2024-01-20T08:58:53.926690+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:58:53.926690+0000","last_undegraded":"2024-01-20T08:58:53.926690+0000","last_fullsized":"2024-01-20T08:58:53.926690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'13","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.531366+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:58:46.531366+0000","last_peered":"2024-01-20T08:58:46.531366+0000","last_clean":"2024-01-20T08:58:46.531366+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:58:46.531366+0000","last_undegraded":"2024-01-20T08:58:46.531366+0000","last_fullsized":"2024-01-20T08:58:46.531366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"27'121","reported_seq":148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.284930+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:58:23.284930+0000","last_peered":"2024-01-20T08:58:23.284930+0000","last_clean":"2024-01-20T08:58:23.284930+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:58:23.284930+0000","last_undegraded":"2024-01-20T08:58:23.284930+0000","last_fullsized":"2024-01-20T08:58:23.284930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'32","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.847454+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:58:25.847454+0000","last_peered":"2024-01-20T08:58:25.847454+0000","last_clean":"2024-01-20T08:58:25.847454+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:58:25.847454+0000","last_undegraded":"2024-01-20T08:58:25.847454+0000","last_fullsized":"2024-01-20T08:58:25.847454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:42.498428+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:58:42.498428+0000","last_peered":"2024-01-20T08:58:42.498428+0000","last_clean":"2024-01-20T08:58:42.498428+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:58:42.498428+0000","last_undegraded":"2024-01-20T08:58:42.498428+0000","last_fullsized":"2024-01-20T08:58:42.498428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'112","reported_seq":139,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:21.538368+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:58:21.538368+0000","last_peered":"2024-01-20T08:58:21.538368+0000","last_clean":"2024-01-20T08:58:21.538368+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:58:21.538368+0000","last_undegraded":"2024-01-20T08:58:21.538368+0000","last_fullsized":"2024-01-20T08:58:21.538368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928171+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:58:53.928171+0000","last_peered":"2024-01-20T08:58:53.928171+0000","last_clean":"2024-01-20T08:58:53.928171+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:58:53.928171+0000","last_undegraded":"2024-01-20T08:58:53.928171+0000","last_fullsized":"2024-01-20T08:58:53.928171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"27'12","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:44.519111+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:58:44.519111+0000","last_peered":"2024-01-20T08:58:44.519111+0000","last_clean":"2024-01-20T08:58:44.519111+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:58:44.519111+0000","last_undegraded":"2024-01-20T08:58:44.519111+0000","last_fullsized":"2024-01-20T08:58:44.519111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.165734+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:58:23.165734+0000","last_peered":"2024-01-20T08:58:23.165734+0000","last_clean":"2024-01-20T08:58:23.165734+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:58:23.165734+0000","last_undegraded":"2024-01-20T08:58:23.165734+0000","last_fullsized":"2024-01-20T08:58:23.165734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:51.115072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929778+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:58:53.929778+0000","last_peered":"2024-01-20T08:58:53.929778+0000","last_clean":"2024-01-20T08:58:53.929778+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:58:53.929778+0000","last_undegraded":"2024-01-20T08:58:53.929778+0000","last_fullsized":"2024-01-20T08:58:53.929778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'315","reported_seq":346,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.274403+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:58:47.274403+0000","last_peered":"2024-01-20T08:58:47.274403+0000","last_clean":"2024-01-20T08:58:47.274403+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:58:47.274403+0000","last_undegraded":"2024-01-20T08:58:47.274403+0000","last_fullsized":"2024-01-20T08:58:47.274403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.953625+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:58:23.953625+0000","last_peered":"2024-01-20T08:58:23.953625+0000","last_clean":"2024-01-20T08:58:23.953625+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:58:23.953625+0000","last_undegraded":"2024-01-20T08:58:23.953625+0000","last_fullsized":"2024-01-20T08:58:23.953625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":11573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.800916+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:58:53.800916+0000","last_peered":"2024-01-20T08:58:53.800916+0000","last_clean":"2024-01-20T08:58:53.800916+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:58:53.800916+0000","last_undegraded":"2024-01-20T08:58:53.800916+0000","last_fullsized":"2024-01-20T08:58:53.800916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.323865+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:58:39.323865+0000","last_peered":"2024-01-20T08:58:39.323865+0000","last_clean":"2024-01-20T08:58:39.323865+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:58:39.323865+0000","last_undegraded":"2024-01-20T08:58:39.323865+0000","last_fullsized":"2024-01-20T08:58:39.323865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"27'126","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.957683+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:58:22.957683+0000","last_peered":"2024-01-20T08:58:22.957683+0000","last_clean":"2024-01-20T08:58:22.957683+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:58:22.957683+0000","last_undegraded":"2024-01-20T08:58:22.957683+0000","last_fullsized":"2024-01-20T08:58:22.957683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892100+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:58:53.892100+0000","last_peered":"2024-01-20T08:58:53.892100+0000","last_clean":"2024-01-20T08:58:53.892100+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:58:53.892100+0000","last_undegraded":"2024-01-20T08:58:53.892100+0000","last_fullsized":"2024-01-20T08:58:53.892100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:59.482554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:43.516881+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:58:43.516881+0000","last_peered":"2024-01-20T08:58:43.516881+0000","last_clean":"2024-01-20T08:58:43.516881+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:58:43.516881+0000","last_undegraded":"2024-01-20T08:58:43.516881+0000","last_fullsized":"2024-01-20T08:58:43.516881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"27'181","reported_seq":209,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.853812+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:58:22.853812+0000","last_peered":"2024-01-20T08:58:22.853812+0000","last_clean":"2024-01-20T08:58:22.853812+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:58:22.853812+0000","last_undegraded":"2024-01-20T08:58:22.853812+0000","last_fullsized":"2024-01-20T08:58:22.853812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019865+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:58:54.019865+0000","last_peered":"2024-01-20T08:58:54.019865+0000","last_clean":"2024-01-20T08:58:54.019865+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:58:54.019865+0000","last_undegraded":"2024-01-20T08:58:54.019865+0000","last_fullsized":"2024-01-20T08:58:54.019865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"27'13","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.315491+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:58:40.315491+0000","last_peered":"2024-01-20T08:58:40.315491+0000","last_clean":"2024-01-20T08:58:40.315491+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:58:40.315491+0000","last_undegraded":"2024-01-20T08:58:40.315491+0000","last_fullsized":"2024-01-20T08:58:40.315491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'113","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.935521+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:58:23.935521+0000","last_peered":"2024-01-20T08:58:23.935521+0000","last_clean":"2024-01-20T08:58:23.935521+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:58:23.935521+0000","last_undegraded":"2024-01-20T08:58:23.935521+0000","last_fullsized":"2024-01-20T08:58:23.935521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808962+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:58:53.808962+0000","last_peered":"2024-01-20T08:58:53.808962+0000","last_clean":"2024-01-20T08:58:53.808962+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:58:53.808962+0000","last_undegraded":"2024-01-20T08:58:53.808962+0000","last_fullsized":"2024-01-20T08:58:53.808962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"27'978","reported_seq":1977,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:49.545029+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:58:49.545029+0000","last_peered":"2024-01-20T08:58:49.545029+0000","last_clean":"2024-01-20T08:58:49.545029+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:58:49.545029+0000","last_undegraded":"2024-01-20T08:58:49.545029+0000","last_fullsized":"2024-01-20T08:58:49.545029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":978,"log_dups_size":0,"ondisk_log_size":978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210232,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":976,"num_write_kb":1034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"27'135","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.812343+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:58:25.812343+0000","last_peered":"2024-01-20T08:58:25.812343+0000","last_clean":"2024-01-20T08:58:25.812343+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:58:25.812343+0000","last_undegraded":"2024-01-20T08:58:25.812343+0000","last_fullsized":"2024-01-20T08:58:25.812343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118,"num_write_kb":10261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"27'28","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804624+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:58:53.804624+0000","last_peered":"2024-01-20T08:58:53.804624+0000","last_clean":"2024-01-20T08:58:53.804624+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:58:53.804624+0000","last_undegraded":"2024-01-20T08:58:53.804624+0000","last_fullsized":"2024-01-20T08:58:53.804624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"27'429","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.345853+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:58:41.345853+0000","last_peered":"2024-01-20T08:58:41.345853+0000","last_clean":"2024-01-20T08:58:41.345853+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:58:41.345853+0000","last_undegraded":"2024-01-20T08:58:41.345853+0000","last_fullsized":"2024-01-20T08:58:41.345853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":4309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"27'129","reported_seq":156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.864992+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:58:23.864992+0000","last_peered":"2024-01-20T08:58:23.864992+0000","last_clean":"2024-01-20T08:58:23.864992+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:58:23.864992+0000","last_undegraded":"2024-01-20T08:58:23.864992+0000","last_fullsized":"2024-01-20T08:58:23.864992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881046+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:58:53.881046+0000","last_peered":"2024-01-20T08:58:53.881046+0000","last_clean":"2024-01-20T08:58:53.881046+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:58:53.881046+0000","last_undegraded":"2024-01-20T08:58:53.881046+0000","last_fullsized":"2024-01-20T08:58:53.881046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"27'9","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.105404+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:58:33.105404+0000","last_peered":"2024-01-20T08:58:33.105404+0000","last_clean":"2024-01-20T08:58:33.105404+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:58:33.105404+0000","last_undegraded":"2024-01-20T08:58:33.105404+0000","last_fullsized":"2024-01-20T08:58:33.105404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:56:51.793427+0000","last_change":"2024-01-20T08:30:47.161637+0000","last_active":"2024-01-20T08:56:51.793427+0000","last_peered":"2024-01-20T08:56:51.793427+0000","last_clean":"2024-01-20T08:56:51.793427+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:56:51.793427+0000","last_undegraded":"2024-01-20T08:56:51.793427+0000","last_fullsized":"2024-01-20T08:56:51.793427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:30:45.183081+0000","last_clean_scrub_stamp":"2024-01-20T08:30:45.183081+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:43:00.370539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'156","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.838096+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:58:24.838096+0000","last_peered":"2024-01-20T08:58:24.838096+0000","last_clean":"2024-01-20T08:58:24.838096+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:58:24.838096+0000","last_undegraded":"2024-01-20T08:58:24.838096+0000","last_fullsized":"2024-01-20T08:58:24.838096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":9711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.880987+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:58:53.880987+0000","last_peered":"2024-01-20T08:58:53.880987+0000","last_clean":"2024-01-20T08:58:53.880987+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:58:53.880987+0000","last_undegraded":"2024-01-20T08:58:53.880987+0000","last_fullsized":"2024-01-20T08:58:53.880987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569355+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:58:45.569355+0000","last_peered":"2024-01-20T08:58:45.569355+0000","last_clean":"2024-01-20T08:58:45.569355+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:58:45.569355+0000","last_undegraded":"2024-01-20T08:58:45.569355+0000","last_fullsized":"2024-01-20T08:58:45.569355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"27'130","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.565862+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:58:23.565862+0000","last_peered":"2024-01-20T08:58:23.565862+0000","last_clean":"2024-01-20T08:58:23.565862+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:58:23.565862+0000","last_undegraded":"2024-01-20T08:58:23.565862+0000","last_fullsized":"2024-01-20T08:58:23.565862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:27.952546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":4819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'39","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.021114+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:58:54.021114+0000","last_peered":"2024-01-20T08:58:54.021114+0000","last_clean":"2024-01-20T08:58:54.021114+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:58:54.021114+0000","last_undegraded":"2024-01-20T08:58:54.021114+0000","last_fullsized":"2024-01-20T08:58:54.021114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.352940+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:58:41.352940+0000","last_peered":"2024-01-20T08:58:41.352940+0000","last_clean":"2024-01-20T08:58:41.352940+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:58:41.352940+0000","last_undegraded":"2024-01-20T08:58:41.352940+0000","last_fullsized":"2024-01-20T08:58:41.352940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"27'188","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.886971+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:58:24.886971+0000","last_peered":"2024-01-20T08:58:24.886971+0000","last_clean":"2024-01-20T08:58:24.886971+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:58:24.886971+0000","last_undegraded":"2024-01-20T08:58:24.886971+0000","last_fullsized":"2024-01-20T08:58:24.886971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'35","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.845305+0000","last_change":"2024-01-20T08:31:44.212803+0000","last_active":"2024-01-20T08:58:53.845305+0000","last_peered":"2024-01-20T08:58:53.845305+0000","last_clean":"2024-01-20T08:58:53.845305+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:58:53.845305+0000","last_undegraded":"2024-01-20T08:58:53.845305+0000","last_fullsized":"2024-01-20T08:58:53.845305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:29.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.295236+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:58:39.295236+0000","last_peered":"2024-01-20T08:58:39.295236+0000","last_clean":"2024-01-20T08:58:39.295236+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:58:39.295236+0000","last_undegraded":"2024-01-20T08:58:39.295236+0000","last_fullsized":"2024-01-20T08:58:39.295236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"27'145","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.862293+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:58:24.862293+0000","last_peered":"2024-01-20T08:58:24.862293+0000","last_clean":"2024-01-20T08:58:24.862293+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:58:24.862293+0000","last_undegraded":"2024-01-20T08:58:24.862293+0000","last_fullsized":"2024-01-20T08:58:24.862293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.893905+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:58:53.893905+0000","last_peered":"2024-01-20T08:58:53.893905+0000","last_clean":"2024-01-20T08:58:53.893905+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:58:53.893905+0000","last_undegraded":"2024-01-20T08:58:53.893905+0000","last_fullsized":"2024-01-20T08:58:53.893905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:12.697455+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:58:12.697455+0000","last_peered":"2024-01-20T08:58:12.697455+0000","last_clean":"2024-01-20T08:58:12.697455+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:58:12.697455+0000","last_undegraded":"2024-01-20T08:58:12.697455+0000","last_fullsized":"2024-01-20T08:58:12.697455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:53:58.454182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"27'160","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.545680+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:58:23.545680+0000","last_peered":"2024-01-20T08:58:23.545680+0000","last_clean":"2024-01-20T08:58:23.545680+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:58:23.545680+0000","last_undegraded":"2024-01-20T08:58:23.545680+0000","last_fullsized":"2024-01-20T08:58:23.545680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:15.192401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":10942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"27'35","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.882452+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:58:53.882452+0000","last_peered":"2024-01-20T08:58:53.882452+0000","last_clean":"2024-01-20T08:58:53.882452+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:58:53.882452+0000","last_undegraded":"2024-01-20T08:58:53.882452+0000","last_fullsized":"2024-01-20T08:58:53.882452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:36.115658+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:58:36.115658+0000","last_peered":"2024-01-20T08:58:36.115658+0000","last_clean":"2024-01-20T08:58:36.115658+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:58:36.115658+0000","last_undegraded":"2024-01-20T08:58:36.115658+0000","last_fullsized":"2024-01-20T08:58:36.115658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"27'142","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.848076+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:58:24.848076+0000","last_peered":"2024-01-20T08:58:24.848076+0000","last_clean":"2024-01-20T08:58:24.848076+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:58:24.848076+0000","last_undegraded":"2024-01-20T08:58:24.848076+0000","last_fullsized":"2024-01-20T08:58:24.848076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881441+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:58:53.881441+0000","last_peered":"2024-01-20T08:58:53.881441+0000","last_clean":"2024-01-20T08:58:53.881441+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:58:53.881441+0000","last_undegraded":"2024-01-20T08:58:53.881441+0000","last_fullsized":"2024-01-20T08:58:53.881441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"27'11","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.570427+0000","last_change":"2024-01-20T08:31:08.583655+0000","last_active":"2024-01-20T08:58:47.570427+0000","last_peered":"2024-01-20T08:58:47.570427+0000","last_clean":"2024-01-20T08:58:47.570427+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:58:47.570427+0000","last_undegraded":"2024-01-20T08:58:47.570427+0000","last_fullsized":"2024-01-20T08:58:47.570427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1169,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"27'133","reported_seq":159,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.549367+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:58:22.549367+0000","last_peered":"2024-01-20T08:58:22.549367+0000","last_clean":"2024-01-20T08:58:22.549367+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:58:22.549367+0000","last_undegraded":"2024-01-20T08:58:22.549367+0000","last_fullsized":"2024-01-20T08:58:22.549367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875228+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:58:53.875228+0000","last_peered":"2024-01-20T08:58:53.875228+0000","last_clean":"2024-01-20T08:58:53.875228+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:58:53.875228+0000","last_undegraded":"2024-01-20T08:58:53.875228+0000","last_fullsized":"2024-01-20T08:58:53.875228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.531270+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:58:45.531270+0000","last_peered":"2024-01-20T08:58:45.531270+0000","last_clean":"2024-01-20T08:58:45.531270+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:58:45.531270+0000","last_undegraded":"2024-01-20T08:58:45.531270+0000","last_fullsized":"2024-01-20T08:58:45.531270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"27'116","reported_seq":142,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.966157+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:58:23.966157+0000","last_peered":"2024-01-20T08:58:23.966157+0000","last_clean":"2024-01-20T08:58:23.966157+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:58:23.966157+0000","last_undegraded":"2024-01-20T08:58:23.966157+0000","last_fullsized":"2024-01-20T08:58:23.966157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"27'40","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.803393+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:58:53.803393+0000","last_peered":"2024-01-20T08:58:53.803393+0000","last_clean":"2024-01-20T08:58:53.803393+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:58:53.803393+0000","last_undegraded":"2024-01-20T08:58:53.803393+0000","last_fullsized":"2024-01-20T08:58:53.803393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"27'111","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.634588+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:58:53.634588+0000","last_peered":"2024-01-20T08:58:53.634588+0000","last_clean":"2024-01-20T08:58:53.634588+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:58:53.634588+0000","last_undegraded":"2024-01-20T08:58:53.634588+0000","last_fullsized":"2024-01-20T08:58:53.634588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069576,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"27'126","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.892559+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:58:23.892559+0000","last_peered":"2024-01-20T08:58:23.892559+0000","last_clean":"2024-01-20T08:58:23.892559+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:58:23.892559+0000","last_undegraded":"2024-01-20T08:58:23.892559+0000","last_fullsized":"2024-01-20T08:58:23.892559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.017745+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:58:54.017745+0000","last_peered":"2024-01-20T08:58:54.017745+0000","last_clean":"2024-01-20T08:58:54.017745+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:58:54.017745+0000","last_undegraded":"2024-01-20T08:58:54.017745+0000","last_fullsized":"2024-01-20T08:58:54.017745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:35.973744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'833","reported_seq":863,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:29.921887+0000","last_change":"2024-01-20T08:31:08.625201+0000","last_active":"2024-01-20T08:58:29.921887+0000","last_peered":"2024-01-20T08:58:29.921887+0000","last_clean":"2024-01-20T08:58:29.921887+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:58:29.921887+0000","last_undegraded":"2024-01-20T08:58:29.921887+0000","last_fullsized":"2024-01-20T08:58:29.921887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:51:46.081620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"27'135","reported_seq":161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.466040+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:58:22.466040+0000","last_peered":"2024-01-20T08:58:22.466040+0000","last_clean":"2024-01-20T08:58:22.466040+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:58:22.466040+0000","last_undegraded":"2024-01-20T08:58:22.466040+0000","last_fullsized":"2024-01-20T08:58:22.466040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881221+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:58:53.881221+0000","last_peered":"2024-01-20T08:58:53.881221+0000","last_clean":"2024-01-20T08:58:53.881221+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:58:53.881221+0000","last_undegraded":"2024-01-20T08:58:53.881221+0000","last_fullsized":"2024-01-20T08:58:53.881221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"27'652","reported_seq":682,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.580377+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:58:47.580377+0000","last_peered":"2024-01-20T08:58:47.580377+0000","last_clean":"2024-01-20T08:58:47.580377+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:58:47.580377+0000","last_undegraded":"2024-01-20T08:58:47.580377+0000","last_fullsized":"2024-01-20T08:58:47.580377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.790975+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:58:24.790975+0000","last_peered":"2024-01-20T08:58:24.790975+0000","last_clean":"2024-01-20T08:58:24.790975+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:58:24.790975+0000","last_undegraded":"2024-01-20T08:58:24.790975+0000","last_fullsized":"2024-01-20T08:58:24.790975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":23255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.833806+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:58:53.833806+0000","last_peered":"2024-01-20T08:58:53.833806+0000","last_clean":"2024-01-20T08:58:53.833806+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:58:53.833806+0000","last_undegraded":"2024-01-20T08:58:53.833806+0000","last_fullsized":"2024-01-20T08:58:53.833806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.102700+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:58:33.102700+0000","last_peered":"2024-01-20T08:58:33.102700+0000","last_clean":"2024-01-20T08:58:33.102700+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:58:33.102700+0000","last_undegraded":"2024-01-20T08:58:33.102700+0000","last_fullsized":"2024-01-20T08:58:33.102700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"27'124","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.864733+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:58:24.864733+0000","last_peered":"2024-01-20T08:58:24.864733+0000","last_clean":"2024-01-20T08:58:24.864733+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:58:24.864733+0000","last_undegraded":"2024-01-20T08:58:24.864733+0000","last_fullsized":"2024-01-20T08:58:24.864733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895614+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:58:53.895614+0000","last_peered":"2024-01-20T08:58:53.895614+0000","last_clean":"2024-01-20T08:58:53.895614+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:58:53.895614+0000","last_undegraded":"2024-01-20T08:58:53.895614+0000","last_fullsized":"2024-01-20T08:58:53.895614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:30:41.788426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.632949+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:58:53.632949+0000","last_peered":"2024-01-20T08:58:53.632949+0000","last_clean":"2024-01-20T08:58:53.632949+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:58:53.632949+0000","last_undegraded":"2024-01-20T08:58:53.632949+0000","last_fullsized":"2024-01-20T08:58:53.632949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.554720+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:58:22.554720+0000","last_peered":"2024-01-20T08:58:22.554720+0000","last_clean":"2024-01-20T08:58:22.554720+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:58:22.554720+0000","last_undegraded":"2024-01-20T08:58:22.554720+0000","last_fullsized":"2024-01-20T08:58:22.554720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"27'130","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.958134+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:58:23.958134+0000","last_peered":"2024-01-20T08:58:23.958134+0000","last_clean":"2024-01-20T08:58:23.958134+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:58:23.958134+0000","last_undegraded":"2024-01-20T08:58:23.958134+0000","last_fullsized":"2024-01-20T08:58:23.958134+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":8002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.932256+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:58:53.932256+0000","last_peered":"2024-01-20T08:58:53.932256+0000","last_clean":"2024-01-20T08:58:53.932256+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:58:53.932256+0000","last_undegraded":"2024-01-20T08:58:53.932256+0000","last_fullsized":"2024-01-20T08:58:53.932256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.360632+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:58:41.360632+0000","last_peered":"2024-01-20T08:58:41.360632+0000","last_clean":"2024-01-20T08:58:41.360632+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:58:41.360632+0000","last_undegraded":"2024-01-20T08:58:41.360632+0000","last_fullsized":"2024-01-20T08:58:41.360632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"27'143","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.710016+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:58:24.710016+0000","last_peered":"2024-01-20T08:58:24.710016+0000","last_clean":"2024-01-20T08:58:24.710016+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:58:24.710016+0000","last_undegraded":"2024-01-20T08:58:24.710016+0000","last_fullsized":"2024-01-20T08:58:24.710016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":18955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'41","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.841394+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:58:53.841394+0000","last_peered":"2024-01-20T08:58:53.841394+0000","last_clean":"2024-01-20T08:58:53.841394+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:58:53.841394+0000","last_undegraded":"2024-01-20T08:58:53.841394+0000","last_fullsized":"2024-01-20T08:58:53.841394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294844+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:58:39.294844+0000","last_peered":"2024-01-20T08:58:39.294844+0000","last_clean":"2024-01-20T08:58:39.294844+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:58:39.294844+0000","last_undegraded":"2024-01-20T08:58:39.294844+0000","last_fullsized":"2024-01-20T08:58:39.294844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"27'675","reported_seq":705,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.711307+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:58:39.711307+0000","last_peered":"2024-01-20T08:58:39.711307+0000","last_clean":"2024-01-20T08:58:39.711307+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:58:39.711307+0000","last_undegraded":"2024-01-20T08:58:39.711307+0000","last_fullsized":"2024-01-20T08:58:39.711307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929004+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:58:53.929004+0000","last_peered":"2024-01-20T08:58:53.929004+0000","last_clean":"2024-01-20T08:58:53.929004+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:58:53.929004+0000","last_undegraded":"2024-01-20T08:58:53.929004+0000","last_fullsized":"2024-01-20T08:58:53.929004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"27'89","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.877290+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:58:23.877290+0000","last_peered":"2024-01-20T08:58:23.877290+0000","last_clean":"2024-01-20T08:58:23.877290+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:58:23.877290+0000","last_undegraded":"2024-01-20T08:58:23.877290+0000","last_fullsized":"2024-01-20T08:58:23.877290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":9333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'689","reported_seq":719,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569769+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:58:45.569769+0000","last_peered":"2024-01-20T08:58:45.569769+0000","last_clean":"2024-01-20T08:58:45.569769+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:58:45.569769+0000","last_undegraded":"2024-01-20T08:58:45.569769+0000","last_fullsized":"2024-01-20T08:58:45.569769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"27'33","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888587+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:58:53.888587+0000","last_peered":"2024-01-20T08:58:53.888587+0000","last_clean":"2024-01-20T08:58:53.888587+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:58:53.888587+0000","last_undegraded":"2024-01-20T08:58:53.888587+0000","last_fullsized":"2024-01-20T08:58:53.888587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"27'147","reported_seq":173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.840471+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:58:24.840471+0000","last_peered":"2024-01-20T08:58:24.840471+0000","last_clean":"2024-01-20T08:58:24.840471+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:58:24.840471+0000","last_undegraded":"2024-01-20T08:58:24.840471+0000","last_fullsized":"2024-01-20T08:58:24.840471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":10430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892176+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:58:53.892176+0000","last_peered":"2024-01-20T08:58:53.892176+0000","last_clean":"2024-01-20T08:58:53.892176+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:58:53.892176+0000","last_undegraded":"2024-01-20T08:58:53.892176+0000","last_fullsized":"2024-01-20T08:58:53.892176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843335+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:58:53.843335+0000","last_peered":"2024-01-20T08:58:53.843335+0000","last_clean":"2024-01-20T08:58:53.843335+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:58:53.843335+0000","last_undegraded":"2024-01-20T08:58:53.843335+0000","last_fullsized":"2024-01-20T08:58:53.843335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"27'44","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875423+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:58:53.875423+0000","last_peered":"2024-01-20T08:58:53.875423+0000","last_clean":"2024-01-20T08:58:53.875423+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:58:53.875423+0000","last_undegraded":"2024-01-20T08:58:53.875423+0000","last_fullsized":"2024-01-20T08:58:53.875423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.839038+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:58:53.839038+0000","last_peered":"2024-01-20T08:58:53.839038+0000","last_clean":"2024-01-20T08:58:53.839038+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:58:53.839038+0000","last_undegraded":"2024-01-20T08:58:53.839038+0000","last_fullsized":"2024-01-20T08:58:53.839038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"27'27","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.807055+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:58:53.807055+0000","last_peered":"2024-01-20T08:58:53.807055+0000","last_clean":"2024-01-20T08:58:53.807055+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:58:53.807055+0000","last_undegraded":"2024-01-20T08:58:53.807055+0000","last_fullsized":"2024-01-20T08:58:53.807055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'31","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.802128+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:58:53.802128+0000","last_peered":"2024-01-20T08:58:53.802128+0000","last_clean":"2024-01-20T08:58:53.802128+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:58:53.802128+0000","last_undegraded":"2024-01-20T08:58:53.802128+0000","last_fullsized":"2024-01-20T08:58:53.802128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"27'44","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895227+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:58:53.895227+0000","last_peered":"2024-01-20T08:58:53.895227+0000","last_clean":"2024-01-20T08:58:53.895227+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:58:53.895227+0000","last_undegraded":"2024-01-20T08:58:53.895227+0000","last_fullsized":"2024-01-20T08:58:53.895227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808904+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:58:53.808904+0000","last_peered":"2024-01-20T08:58:53.808904+0000","last_clean":"2024-01-20T08:58:53.808904+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:58:53.808904+0000","last_undegraded":"2024-01-20T08:58:53.808904+0000","last_fullsized":"2024-01-20T08:58:53.808904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.871511+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:58:53.871511+0000","last_peered":"2024-01-20T08:58:53.871511+0000","last_clean":"2024-01-20T08:58:53.871511+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:58:53.871511+0000","last_undegraded":"2024-01-20T08:58:53.871511+0000","last_fullsized":"2024-01-20T08:58:53.871511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"27'42","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.837960+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:58:53.837960+0000","last_peered":"2024-01-20T08:58:53.837960+0000","last_clean":"2024-01-20T08:58:53.837960+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:58:53.837960+0000","last_undegraded":"2024-01-20T08:58:53.837960+0000","last_fullsized":"2024-01-20T08:58:53.837960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"27'38","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888654+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:58:53.888654+0000","last_peered":"2024-01-20T08:58:53.888654+0000","last_clean":"2024-01-20T08:58:53.888654+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:58:53.888654+0000","last_undegraded":"2024-01-20T08:58:53.888654+0000","last_fullsized":"2024-01-20T08:58:53.888654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:27.367997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'47","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928357+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:58:53.928357+0000","last_peered":"2024-01-20T08:58:53.928357+0000","last_clean":"2024-01-20T08:58:53.928357+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:58:53.928357+0000","last_undegraded":"2024-01-20T08:58:53.928357+0000","last_fullsized":"2024-01-20T08:58:53.928357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.021424+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:58:54.021424+0000","last_peered":"2024-01-20T08:58:54.021424+0000","last_clean":"2024-01-20T08:58:54.021424+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:58:54.021424+0000","last_undegraded":"2024-01-20T08:58:54.021424+0000","last_fullsized":"2024-01-20T08:58:54.021424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878349+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:58:53.878349+0000","last_peered":"2024-01-20T08:58:53.878349+0000","last_clean":"2024-01-20T08:58:53.878349+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:58:53.878349+0000","last_undegraded":"2024-01-20T08:58:53.878349+0000","last_fullsized":"2024-01-20T08:58:53.878349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"27'45","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.846302+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:58:53.846302+0000","last_peered":"2024-01-20T08:58:53.846302+0000","last_clean":"2024-01-20T08:58:53.846302+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:58:53.846302+0000","last_undegraded":"2024-01-20T08:58:53.846302+0000","last_fullsized":"2024-01-20T08:58:53.846302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"27'38","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.870937+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:58:53.870937+0000","last_peered":"2024-01-20T08:58:53.870937+0000","last_clean":"2024-01-20T08:58:53.870937+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:58:53.870937+0000","last_undegraded":"2024-01-20T08:58:53.870937+0000","last_fullsized":"2024-01-20T08:58:53.870937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881740+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:58:53.881740+0000","last_peered":"2024-01-20T08:58:53.881740+0000","last_clean":"2024-01-20T08:58:53.881740+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:58:53.881740+0000","last_undegraded":"2024-01-20T08:58:53.881740+0000","last_fullsized":"2024-01-20T08:58:53.881740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.834894+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:58:53.834894+0000","last_peered":"2024-01-20T08:58:53.834894+0000","last_clean":"2024-01-20T08:58:53.834894+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:58:53.834894+0000","last_undegraded":"2024-01-20T08:58:53.834894+0000","last_fullsized":"2024-01-20T08:58:53.834894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804360+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:58:53.804360+0000","last_peered":"2024-01-20T08:58:53.804360+0000","last_clean":"2024-01-20T08:58:53.804360+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:58:53.804360+0000","last_undegraded":"2024-01-20T08:58:53.804360+0000","last_fullsized":"2024-01-20T08:58:53.804360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806862+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:58:53.806862+0000","last_peered":"2024-01-20T08:58:53.806862+0000","last_clean":"2024-01-20T08:58:53.806862+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:58:53.806862+0000","last_undegraded":"2024-01-20T08:58:53.806862+0000","last_fullsized":"2024-01-20T08:58:53.806862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023455+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:58:54.023455+0000","last_peered":"2024-01-20T08:58:54.023455+0000","last_clean":"2024-01-20T08:58:54.023455+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:58:54.023455+0000","last_undegraded":"2024-01-20T08:58:54.023455+0000","last_fullsized":"2024-01-20T08:58:54.023455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022954+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:58:54.022954+0000","last_peered":"2024-01-20T08:58:54.022954+0000","last_clean":"2024-01-20T08:58:54.022954+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:58:54.022954+0000","last_undegraded":"2024-01-20T08:58:54.022954+0000","last_fullsized":"2024-01-20T08:58:54.022954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023780+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:58:54.023780+0000","last_peered":"2024-01-20T08:58:54.023780+0000","last_clean":"2024-01-20T08:58:54.023780+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:58:54.023780+0000","last_undegraded":"2024-01-20T08:58:54.023780+0000","last_fullsized":"2024-01-20T08:58:54.023780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.889696+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:58:53.889696+0000","last_peered":"2024-01-20T08:58:53.889696+0000","last_clean":"2024-01-20T08:58:53.889696+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:58:53.889696+0000","last_undegraded":"2024-01-20T08:58:53.889696+0000","last_fullsized":"2024-01-20T08:58:53.889696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843441+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:58:53.843441+0000","last_peered":"2024-01-20T08:58:53.843441+0000","last_clean":"2024-01-20T08:58:53.843441+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:58:53.843441+0000","last_undegraded":"2024-01-20T08:58:53.843441+0000","last_fullsized":"2024-01-20T08:58:53.843441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.017192+0000","last_change":"2024-01-20T08:31:10.197708+0000","last_active":"2024-01-20T08:58:54.017192+0000","last_peered":"2024-01-20T08:58:54.017192+0000","last_clean":"2024-01-20T08:58:54.017192+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:58:54.017192+0000","last_undegraded":"2024-01-20T08:58:54.017192+0000","last_fullsized":"2024-01-20T08:58:54.017192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892312+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:58:53.892312+0000","last_peered":"2024-01-20T08:58:53.892312+0000","last_clean":"2024-01-20T08:58:53.892312+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:58:53.892312+0000","last_undegraded":"2024-01-20T08:58:53.892312+0000","last_fullsized":"2024-01-20T08:58:53.892312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:12.931970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"27'30","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019038+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:58:54.019038+0000","last_peered":"2024-01-20T08:58:54.019038+0000","last_clean":"2024-01-20T08:58:54.019038+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:58:54.019038+0000","last_undegraded":"2024-01-20T08:58:54.019038+0000","last_fullsized":"2024-01-20T08:58:54.019038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'25","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892157+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:58:53.892157+0000","last_peered":"2024-01-20T08:58:53.892157+0000","last_clean":"2024-01-20T08:58:53.892157+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:58:53.892157+0000","last_undegraded":"2024-01-20T08:58:53.892157+0000","last_fullsized":"2024-01-20T08:58:53.892157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:16:50.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'41","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.842487+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:58:53.842487+0000","last_peered":"2024-01-20T08:58:53.842487+0000","last_clean":"2024-01-20T08:58:53.842487+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:58:53.842487+0000","last_undegraded":"2024-01-20T08:58:53.842487+0000","last_fullsized":"2024-01-20T08:58:53.842487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806164+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:58:53.806164+0000","last_peered":"2024-01-20T08:58:53.806164+0000","last_clean":"2024-01-20T08:58:53.806164+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:58:53.806164+0000","last_undegraded":"2024-01-20T08:58:53.806164+0000","last_fullsized":"2024-01-20T08:58:53.806164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"27'26","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022116+0000","last_change":"2024-01-20T08:31:10.234003+0000","last_active":"2024-01-20T08:58:54.022116+0000","last_peered":"2024-01-20T08:58:54.022116+0000","last_clean":"2024-01-20T08:58:54.022116+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:58:54.022116+0000","last_undegraded":"2024-01-20T08:58:54.022116+0000","last_fullsized":"2024-01-20T08:58:54.022116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":308552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4420,"ondisk_log_size":4420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39037201,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":8593,"num_write_kb":43222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78323712,"data_stored":78074402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8116,"ondisk_log_size":8116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509784,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39732,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332108,"statfs":{"total":96636764160,"available":96596078592,"internally_reserved":0,"allocated":4501504,"data_stored":4315319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46999999999999997,"15min":0.45300000000000001},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.23899999999999999},"max":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.25800000000000001},"max":{"1min":0.60999999999999999,"5min":0.66200000000000003,"15min":0.81999999999999995},"last":0.46999999999999997}]},{"osd":1,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.67900000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28999999999999998},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.67500000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.70899999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.628,"5min":0.68500000000000005,"15min":0.72699999999999998},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.30299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.74199999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44800000000000001,"15min":0.45400000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.23799999999999999},"max":{"1min":0.63500000000000001,"5min":0.67400000000000004,"15min":0.76700000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75800000000000001,"15min":0.876},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.81000000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.79400000000000004},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.27600000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.90900000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"last":0.54400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424509784,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67036,"kb_used_data":13652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94304804,"statfs":{"total":96636764160,"available":96568119296,"internally_reserved":0,"allocated":13979648,"data_stored":13773597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.29099999999999998},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17199999999999999},"max":{"1min":0.41899999999999998,"5min":0.441,"15min":0.72599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.19500000000000001,"15min":0.19400000000000001},"max":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.58799999999999997},"last":0.28999999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.503,"15min":0.61799999999999999},"last":0.315},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.216,"15min":0.21099999999999999},"max":{"1min":0.39500000000000002,"5min":0.51000000000000001,"15min":0.79600000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.69499999999999995},"last":0.37}]},{"osd":4,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.72599999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.77600000000000002,"15min":0.91400000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.48499999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.78500000000000003},"last":0.497}]}]},{"osd":5,"up_from":14,"seq":60129542486,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70924,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300916,"statfs":{"total":96636764160,"available":96564137984,"internally_reserved":0,"allocated":13701120,"data_stored":13502280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"max":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.69599999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72099999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.66000000000000003},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.72899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.76700000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.80900000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44,"15min":0.439},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48399999999999999,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.26500000000000001},"max":{"1min":0.56799999999999995,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.81599999999999995},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.80300000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59399999999999997,"5min":0.76200000000000001,"15min":0.85799999999999998},"last":0.44400000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542485,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66068,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305772,"statfs":{"total":96636764160,"available":96569110528,"internally_reserved":0,"allocated":8728576,"data_stored":8508591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.61299999999999999,"15min":0.70199999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.629,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.31}]},{"osd":2,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.31,"15min":0.26500000000000001},"max":{"1min":0.51400000000000001,"5min":0.69599999999999995,"15min":0.73299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.315,"5min":0.31,"15min":0.28599999999999998},"max":{"1min":0.54700000000000004,"5min":0.622,"15min":0.66700000000000004},"last":0.38800000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57799999999999996,"5min":0.71899999999999997,"15min":0.73999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.254},"max":{"1min":0.61899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.26000000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.59099999999999997,"5min":0.75700000000000001,"15min":0.78400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":0.75600000000000001},"last":0.27200000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66000000000000003,"5min":0.78300000000000003,"15min":0.79100000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":0.71699999999999997,"15min":0.79000000000000004},"last":0.28899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542486,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43824,"kb_used_data":8488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328016,"statfs":{"total":96636764160,"available":96591888384,"internally_reserved":0,"allocated":8691712,"data_stored":8506821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.45800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29999999999999999,"15min":0.307},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":0.38100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.26300000000000001}]},{"osd":1,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.502,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.308}]},{"osd":3,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33000000000000002,"15min":0.33200000000000002},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.378,"5min":0.55900000000000005,"15min":0.60799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34599999999999997}]},{"osd":4,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.71399999999999997,"15min":0.82099999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.376}]},{"osd":5,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.86699999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.80500000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.69999999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.76000000000000001,"15min":0.79300000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.36399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85299999999999998},"last":0.439}]}]},{"osd":1,"up_from":14,"seq":60129542487,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9289728,"data_stored":9097863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30099999999999999,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.55700000000000005,"15min":0.59499999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.32000000000000001,"15min":0.31},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20000000000000001},"max":{"1min":0.40600000000000003,"5min":0.45500000000000002,"15min":0.54400000000000004},"last":0.307}]},{"osd":2,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.308,"15min":0.314},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.35399999999999998,"5min":0.47999999999999998,"15min":0.49199999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":0.45800000000000002,"5min":0.624,"15min":0.624},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.59399999999999997,"15min":0.63},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.44700000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.27900000000000003},"max":{"1min":0.80700000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39300000000000002}]},{"osd":5,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.876},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":0.873,"15min":0.873},"last":0.49199999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46400000000000002}]}]},{"osd":7,"up_from":13,"seq":55834575190,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63212,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308628,"statfs":{"total":96636764160,"available":96572035072,"internally_reserved":0,"allocated":5804032,"data_stored":5592459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.68999999999999995},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.54200000000000004,"5min":0.65100000000000002,"15min":0.73499999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.37,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.78000000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23000000000000001},"max":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.75800000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26600000000000001},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.79900000000000004},"last":0.40400000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.76400000000000001,"15min":0.77800000000000002},"last":0.443}]},{"osd":6,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.371},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.52200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575190,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61772,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310068,"statfs":{"total":96636764160,"available":96573509632,"internally_reserved":0,"allocated":17108992,"data_stored":16897992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.505,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.31,"15min":0.30399999999999999},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.41699999999999998,"5min":0.47599999999999998,"15min":0.51200000000000001},"last":0.30399999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47199999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.42299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29399999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44700000000000001,"5min":0.625,"15min":0.625},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.373,"15min":0.371},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.434,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.78500000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.71399999999999997,"5min":0.80000000000000004,"15min":0.85499999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.76100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.34000000000000002},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.497},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13692928,"data_stored":13656124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8409088,"data_stored":8390022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":8389348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16822272,"data_stored":16780519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8441856,"data_stored":8391118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12820480,"data_stored":12794439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4214784,"data_stored":4197846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":5474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:59:01.757 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:59:01.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.891+0000 7fe0cc189640 1 Processor -- start 2024-01-20T08:59:01.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.892+0000 7fe0cc189640 1 -- start start 2024-01-20T08:59:01.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.892+0000 7fe0cc189640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0c4070cf0 0x7fe0c40c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.892+0000 7fe0cc189640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 0x7fe0c4157b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.892+0000 7fe0cc189640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c415ba40 con 0x7fe0c40ca260 2024-01-20T08:59:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.892+0000 7fe0cc189640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40c3bb0 con 0x7fe0c4070cf0 2024-01-20T08:59:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.892+0000 7fe0cc189640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c96f0 con 0x7fe0c40710d0 2024-01-20T08:59:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.893+0000 7fe0c9724640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 0x7fe0c4157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.893+0000 7fe0c9724640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 0x7fe0c4157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50148/0 (socket says 172.21.15.87:50148) 2024-01-20T08:59:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.893+0000 7fe0c9724640 1 -- 172.21.15.87:0/3735630696 learned_addr learned my addr 172.21.15.87:0/3735630696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.893+0000 7fe0c9f25640 1 --2- 172.21.15.87:0/3735630696 >> v2:172.21.15.116:3300/0 conn(0x7fe0c4070cf0 0x7fe0c40c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.895+0000 7fe0c9724640 1 -- 172.21.15.87:0/3735630696 >> v2:172.21.15.116:3300/0 conn(0x7fe0c4070cf0 msgr2=0x7fe0c40c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:01.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.895+0000 7fe0c9724640 1 --2- 172.21.15.87:0/3735630696 >> v2:172.21.15.116:3300/0 conn(0x7fe0c4070cf0 0x7fe0c40c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.895+0000 7fe0c9724640 1 -- 172.21.15.87:0/3735630696 >> v1:172.21.15.116:6789/0 conn(0x7fe0c40ca260 legacy=0x7fe0c4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.895+0000 7fe0c9724640 1 -- 172.21.15.87:0/3735630696 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c41576e0 con 0x7fe0c40710d0 2024-01-20T08:59:01.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.898+0000 7fe0c9724640 1 --2- 172.21.15.87:0/3735630696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 0x7fe0c4157b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009870 tx=0x7fe0b402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ca6619d6c4499c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:01.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0c8f23640 1 -- 172.21.15.87:0/3735630696 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b402f840 con 0x7fe0c40710d0 2024-01-20T08:59:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0c8f23640 1 -- 172.21.15.87:0/3735630696 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b402f9a0 con 0x7fe0c40710d0 2024-01-20T08:59:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0c8f23640 1 -- 172.21.15.87:0/3735630696 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b402fd30 con 0x7fe0c40710d0 2024-01-20T08:59:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 -- 172.21.15.87:0/3735630696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 msgr2=0x7fe0c4157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 --2- 172.21.15.87:0/3735630696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 0x7fe0c4157b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009870 tx=0x7fe0b402ba20 comp rx=0 tx=0).stop 2024-01-20T08:59:01.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 -- 172.21.15.87:0/3735630696 shutdown_connections 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 --2- 172.21.15.87:0/3735630696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40710d0 0x7fe0c4157b70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 --2- 172.21.15.87:0/3735630696 >> v2:172.21.15.116:3300/0 conn(0x7fe0c4070cf0 0x7fe0c40c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 -- 172.21.15.87:0/3735630696 >> 172.21.15.87:0/3735630696 conn(0x7fe0c4066ac0 msgr2=0x7fe0c406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 -- 172.21.15.87:0/3735630696 shutdown_connections 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.902+0000 7fe0cc189640 1 -- 172.21.15.87:0/3735630696 wait complete. 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.903+0000 7fe0cc189640 1 Processor -- start 2024-01-20T08:59:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 -- start start 2024-01-20T08:59:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c40710d0 0x7fe0c41e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0c40ca260 0x7fe0c4157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 0x7fe0c41ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c41e2e30 con 0x7fe0c40ca260 2024-01-20T08:59:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c41e35e0 con 0x7fe0c40710d0 2024-01-20T08:59:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.904+0000 7fe0cc189640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c41f0930 con 0x7fe0c40cbff0 2024-01-20T08:59:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.905+0000 7fe0c9724640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0c40ca260 0x7fe0c4157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.905+0000 7fe0c9f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c40710d0 0x7fe0c41e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.905+0000 7fe0ca726640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 0x7fe0c41ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.905+0000 7fe0ca726640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 0x7fe0c41ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50156/0 (socket says 172.21.15.87:50156) 2024-01-20T08:59:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.905+0000 7fe0ca726640 1 -- 172.21.15.87:0/1305441663 learned_addr learned my addr 172.21.15.87:0/1305441663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:01.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.909+0000 7fe0c9f25640 1 -- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 msgr2=0x7fe0c41ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.909+0000 7fe0c9f25640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 0x7fe0c41ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.909+0000 7fe0c9f25640 1 -- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0c40ca260 msgr2=0x7fe0c4157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.909+0000 7fe0c9f25640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0c40ca260 0x7fe0c4157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.909+0000 7fe0c9f25640 1 -- 172.21.15.87:0/1305441663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40094d0 con 0x7fe0c40710d0 2024-01-20T08:59:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.909+0000 7fe0ca726640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 0x7fe0c41ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.911+0000 7fe0c9f25640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c40710d0 0x7fe0c41e43f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000e880 tx=0x7fe0c000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:01.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.914+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000c940 con 0x7fe0c40710d0 2024-01-20T08:59:01.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.914+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c000caa0 con 0x7fe0c40710d0 2024-01-20T08:59:01.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.914+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c41f0bd0 con 0x7fe0c40710d0 2024-01-20T08:59:01.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.914+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000ce50 con 0x7fe0c40710d0 2024-01-20T08:59:01.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.914+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c41f1190 con 0x7fe0c40710d0 2024-01-20T08:59:01.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.916+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c40b2f70 con 0x7fe0c40710d0 2024-01-20T08:59:01.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.934+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe0c0013470 con 0x7fe0c40710d0 2024-01-20T08:59:01.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.935+0000 7fe0ba7fc640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0a806af20 0x7fe0a806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:01.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.935+0000 7fe0c9724640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0a806af20 0x7fe0a806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:01.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:01.936+0000 7fe0c9724640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0a806af20 0x7fe0a806d3e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4036000 tx=0x7fe0b4008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:02.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.167+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fe0c0058b40 con 0x7fe0c40710d0 2024-01-20T08:59:02.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.190+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe0c0054e80 con 0x7fe0c40710d0 2024-01-20T08:59:02.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.368+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0c4067680 con 0x7fe0c40710d0 2024-01-20T08:59:02.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.405+0000 7fe0ba7fc640 1 -- 172.21.15.87:0/1305441663 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7fe0c0021020 con 0x7fe0c40710d0 2024-01-20T08:59:02.408 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:02.408 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:59:02.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0a806af20 msgr2=0x7fe0a806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0a806af20 0x7fe0a806d3e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4036000 tx=0x7fe0b4008030 comp rx=0 tx=0).stop 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c40710d0 msgr2=0x7fe0c41e43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c40710d0 0x7fe0c41e43f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000e880 tx=0x7fe0c000ed40 comp rx=0 tx=0).stop 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 shutdown_connections 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe0a806af20 0x7fe0a806d3e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0c40cbff0 0x7fe0c41ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0c40ca260 0x7fe0c4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 --2- 172.21.15.87:0/1305441663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c40710d0 0x7fe0c41e43f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 >> 172.21.15.87:0/1305441663 conn(0x7fe0c4070cf0 msgr2=0x7fe0c4159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 shutdown_connections 2024-01-20T08:59:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:02.407+0000 7fe0cc189640 1 -- 172.21.15.87:0/1305441663 wait complete. 2024-01-20T08:59:03.421 INFO:tasks.ceph:Scrubbing osd.0 2024-01-20T08:59:03.421 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:03.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.527+0000 7fc1a0096640 1 Processor -- start 2024-01-20T08:59:03.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.527+0000 7fc1a0096640 1 -- start start 2024-01-20T08:59:03.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.528+0000 7fc1a0096640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc198072920 0x7fc19806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.528+0000 7fc1a0096640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc198071c80 con 0x7fc198157b50 2024-01-20T08:59:03.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.528+0000 7fc1a0096640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19815dac0 con 0x7fc198161380 2024-01-20T08:59:03.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.528+0000 7fc1a0096640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19815dda0 con 0x7fc198072920 2024-01-20T08:59:03.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.528+0000 7fc19de32640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc198157b50 0x7fc1980cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34180/0 (socket says 172.21.15.87:34180) 2024-01-20T08:59:03.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.528+0000 7fc19de32640 1 -- 172.21.15.87:0/3947737077 learned_addr learned my addr 172.21.15.87:0/3947737077 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:03.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.529+0000 7fc19d631640 1 --2- 172.21.15.87:0/3947737077 >> v2:172.21.15.116:3300/0 conn(0x7fc198072920 0x7fc19806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.531+0000 7fc19ce30640 1 -- 172.21.15.87:0/3947737077 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024032534 0 0) 0x7fc198071c80 con 0x7fc198157b50 2024-01-20T08:59:03.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.531+0000 7fc19ce30640 1 -- 172.21.15.87:0/3947737077 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc184003520 con 0x7fc198157b50 2024-01-20T08:59:03.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.532+0000 7fc19d631640 1 -- 172.21.15.87:0/3947737077 >> v1:172.21.15.116:6789/0 conn(0x7fc198161380 legacy=0x7fc1981617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.532+0000 7fc19d631640 1 -- 172.21.15.87:0/3947737077 >> v1:172.21.15.87:6789/0 conn(0x7fc198157b50 legacy=0x7fc1980cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.532+0000 7fc19d631640 1 -- 172.21.15.87:0/3947737077 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19815e080 con 0x7fc198072920 2024-01-20T08:59:03.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.535+0000 7fc19d631640 1 --2- 172.21.15.87:0/3947737077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198072920 0x7fc19806f3e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1880097d0 tx=0x7fc18802b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913c47fed446ad3b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.540+0000 7fc19ce30640 1 -- 172.21.15.87:0/3947737077 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188030890 con 0x7fc198072920 2024-01-20T08:59:03.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.540+0000 7fc19ce30640 1 -- 172.21.15.87:0/3947737077 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1880309f0 con 0x7fc198072920 2024-01-20T08:59:03.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.540+0000 7fc19ce30640 1 -- 172.21.15.87:0/3947737077 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188030d80 con 0x7fc198072920 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 -- 172.21.15.87:0/3947737077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198072920 msgr2=0x7fc19806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 --2- 172.21.15.87:0/3947737077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198072920 0x7fc19806f3e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1880097d0 tx=0x7fc18802b9d0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 -- 172.21.15.87:0/3947737077 shutdown_connections 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 --2- 172.21.15.87:0/3947737077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198072920 0x7fc19806f3e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 -- 172.21.15.87:0/3947737077 >> 172.21.15.87:0/3947737077 conn(0x7fc19806d510 msgr2=0x7fc1980663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 -- 172.21.15.87:0/3947737077 shutdown_connections 2024-01-20T08:59:03.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.541+0000 7fc1a0096640 1 -- 172.21.15.87:0/3947737077 wait complete. 2024-01-20T08:59:03.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.542+0000 7fc1a0096640 1 Processor -- start 2024-01-20T08:59:03.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.542+0000 7fc1a0096640 1 -- start start 2024-01-20T08:59:03.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc1a0096640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc198072920 0x7fc1981f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc1a0096640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198157b50 0x7fc19815f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc1a0096640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc198161380 0x7fc19814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc1a0096640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198152d80 con 0x7fc198161380 2024-01-20T08:59:03.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc1a0096640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19815ed80 con 0x7fc198157b50 2024-01-20T08:59:03.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc1a0096640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc198155350 con 0x7fc198072920 2024-01-20T08:59:03.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.543+0000 7fc19e633640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc198161380 0x7fc19814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.544+0000 7fc19e633640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc198161380 0x7fc19814f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60044/0 (socket says 172.21.15.87:60044) 2024-01-20T08:59:03.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.544+0000 7fc19e633640 1 -- 172.21.15.87:0/2116221458 learned_addr learned my addr 172.21.15.87:0/2116221458 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:03.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.544+0000 7fc19d631640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198157b50 0x7fc19815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.544+0000 7fc19de32640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc198072920 0x7fc1981f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.547+0000 7fc19e633640 1 -- 172.21.15.87:0/2116221458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc198072920 msgr2=0x7fc1981f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.547+0000 7fc19e633640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc198072920 0x7fc1981f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.547+0000 7fc19e633640 1 -- 172.21.15.87:0/2116221458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198157b50 msgr2=0x7fc19815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.547+0000 7fc19e633640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc198157b50 0x7fc19815f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.547+0000 7fc19e633640 1 -- 172.21.15.87:0/2116221458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188009400 con 0x7fc198161380 2024-01-20T08:59:03.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.549+0000 7fc19e633640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc198161380 0x7fc19814f1d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc19000c730 tx=0x7fc19000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.554+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190006d50 con 0x7fc198161380 2024-01-20T08:59:03.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.554+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc190003970 con 0x7fc198161380 2024-01-20T08:59:03.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.554+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190003d20 con 0x7fc198161380 2024-01-20T08:59:03.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.554+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1981f7a90 con 0x7fc198161380 2024-01-20T08:59:03.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.554+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981523e0 con 0x7fc198161380 2024-01-20T08:59:03.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.555+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1980c6000 con 0x7fc198161380 2024-01-20T08:59:03.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.575+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc1900050d0 con 0x7fc198161380 2024-01-20T08:59:03.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.575+0000 7fc18e7fc640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc17c06af20 0x7fc17c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.575+0000 7fc19de32640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc17c06af20 0x7fc17c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.576+0000 7fc19de32640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc17c06af20 0x7fc17c06d3e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc194005880 tx=0x7fc194009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.578+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fc1900587d0 con 0x7fc198161380 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.578+0000 7fc18e7fc640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7fc17c0745a0 0x7fc17c076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.579+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 --> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc17c0770f0 con 0x7fc17c0745a0 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.579+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc190054b10 con 0x7fc198161380 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.579+0000 7fc19d631640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7fc17c0745a0 0x7fc17c076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.579+0000 7fc19d631640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7fc17c0745a0 0x7fc17c076a20 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.580+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== osd.0 v2:172.21.15.87:6800/1449648137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc17c0770f0 con 0x7fc17c0745a0 2024-01-20T08:59:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.593+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 --> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc19815e640 con 0x7fc17c0745a0 2024-01-20T08:59:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.593+0000 7fc18e7fc640 1 -- 172.21.15.87:0/2116221458 <== osd.0 v2:172.21.15.87:6800/1449648137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc188030b60 con 0x7fc17c0745a0 2024-01-20T08:59:03.594 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:03.594 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:03.594 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:03.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.593+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7fc17c0745a0 msgr2=0x7fc17c076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.593+0000 7fc1a0096640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6800/1449648137,v1:172.21.15.87:6801/1449648137] conn(0x7fc17c0745a0 0x7fc17c076a20 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.593+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc17c06af20 msgr2=0x7fc17c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.593+0000 7fc1a0096640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fc17c06af20 0x7fc17c06d3e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc194005880 tx=0x7fc194009040 comp rx=0 tx=0).stop 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc198161380 msgr2=0x7fc19814f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc1a0096640 1 --2- 172.21.15.87:0/2116221458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc198161380 0x7fc19814f1d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc19000c730 tx=0x7fc19000cb30 comp rx=0 tx=0).stop 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc19e633640 1 -- 172.21.15.87:0/2116221458 reap_dead start 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 shutdown_connections 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 >> 172.21.15.87:0/2116221458 conn(0x7fc19806d510 msgr2=0x7fc198150170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 shutdown_connections 2024-01-20T08:59:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.594+0000 7fc1a0096640 1 -- 172.21.15.87:0/2116221458 wait complete. 2024-01-20T08:59:03.605 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-20T08:59:03.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 Processor -- start 2024-01-20T08:59:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 -- start start 2024-01-20T08:59:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b141517f0 0x7f6b140b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c90e0 0x7f6b14157460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 0x7f6b140ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b140b0520 con 0x7f6b141517f0 2024-01-20T08:59:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b14151bd0 con 0x7f6b140c9520 2024-01-20T08:59:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.706+0000 7f6b19f10640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140ca070 con 0x7f6b140c90e0 2024-01-20T08:59:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.707+0000 7f6b13fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 0x7f6b140ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.707+0000 7f6b0affd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c90e0 0x7f6b14157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.707+0000 7f6b137fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b141517f0 0x7f6b140b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.707+0000 7f6b13fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 0x7f6b140ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58116/0 (socket says 172.21.15.87:58116) 2024-01-20T08:59:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.707+0000 7f6b13fff640 1 -- 172.21.15.87:0/2764892177 learned_addr learned my addr 172.21.15.87:0/2764892177 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.710+0000 7f6b0affd640 1 -- 172.21.15.87:0/2764892177 >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 msgr2=0x7f6b140ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.710+0000 7f6b0affd640 1 --2- 172.21.15.87:0/2764892177 >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 0x7f6b140ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.710+0000 7f6b0affd640 1 -- 172.21.15.87:0/2764892177 >> v2:172.21.15.87:3300/0 conn(0x7f6b141517f0 msgr2=0x7f6b140b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.710+0000 7f6b0affd640 1 --2- 172.21.15.87:0/2764892177 >> v2:172.21.15.87:3300/0 conn(0x7f6b141517f0 0x7f6b140b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.710+0000 7f6b0affd640 1 -- 172.21.15.87:0/2764892177 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b14157ef0 con 0x7f6b140c90e0 2024-01-20T08:59:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.710+0000 7f6b13fff640 1 --2- 172.21.15.87:0/2764892177 >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 0x7f6b140ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.712+0000 7f6b0affd640 1 --2- 172.21.15.87:0/2764892177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c90e0 0x7f6b14157460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b140b2d60 tx=0x7f6afc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b6b42d31057f11 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.715+0000 7f6b12ffd640 1 -- 172.21.15.87:0/2764892177 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc02e070 con 0x7f6b140c90e0 2024-01-20T08:59:03.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b12ffd640 1 -- 172.21.15.87:0/2764892177 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0315a0 con 0x7f6b140c90e0 2024-01-20T08:59:03.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b12ffd640 1 -- 172.21.15.87:0/2764892177 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc031930 con 0x7f6b140c90e0 2024-01-20T08:59:03.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 -- 172.21.15.87:0/2764892177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c90e0 msgr2=0x7f6b14157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 --2- 172.21.15.87:0/2764892177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c90e0 0x7f6b14157460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b140b2d60 tx=0x7f6afc02b840 comp rx=0 tx=0).stop 2024-01-20T08:59:03.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 -- 172.21.15.87:0/2764892177 shutdown_connections 2024-01-20T08:59:03.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 --2- 172.21.15.87:0/2764892177 >> v2:172.21.15.116:3300/0 conn(0x7f6b140c9520 0x7f6b140ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 --2- 172.21.15.87:0/2764892177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c90e0 0x7f6b14157460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 --2- 172.21.15.87:0/2764892177 >> v2:172.21.15.87:3300/0 conn(0x7f6b141517f0 0x7f6b140b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 -- 172.21.15.87:0/2764892177 >> 172.21.15.87:0/2764892177 conn(0x7f6b1406a5b0 msgr2=0x7f6b1406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 -- 172.21.15.87:0/2764892177 shutdown_connections 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.716+0000 7f6b19f10640 1 -- 172.21.15.87:0/2764892177 wait complete. 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.717+0000 7f6b19f10640 1 Processor -- start 2024-01-20T08:59:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 -- start start 2024-01-20T08:59:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b140c90e0 0x7f6b141f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c9520 0x7f6b14159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 0x7f6b141598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b141f1910 con 0x7f6b141517f0 2024-01-20T08:59:03.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b14158bf0 con 0x7f6b140c90e0 2024-01-20T08:59:03.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.718+0000 7f6b19f10640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b141f1630 con 0x7f6b140c9520 2024-01-20T08:59:03.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.719+0000 7f6b13fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 0x7f6b141598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.719+0000 7f6b0affd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c9520 0x7f6b14159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.719+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b140c90e0 0x7f6b141f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.719+0000 7f6b13fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 0x7f6b141598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60060/0 (socket says 172.21.15.87:60060) 2024-01-20T08:59:03.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.719+0000 7f6b13fff640 1 -- 172.21.15.87:0/1938172732 learned_addr learned my addr 172.21.15.87:0/1938172732 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.723+0000 7f6b13fff640 1 -- 172.21.15.87:0/1938172732 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c9520 msgr2=0x7f6b14159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.723+0000 7f6b13fff640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c9520 0x7f6b14159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.723+0000 7f6b13fff640 1 -- 172.21.15.87:0/1938172732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b140c90e0 msgr2=0x7f6b141f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.723+0000 7f6b13fff640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b140c90e0 0x7f6b141f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.723+0000 7f6b13fff640 1 -- 172.21.15.87:0/1938172732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009b30 con 0x7f6b141517f0 2024-01-20T08:59:03.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.725+0000 7f6b13fff640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 0x7f6b141598b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00eed0 tx=0x7f6b0c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.728+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c010040 con 0x7f6b141517f0 2024-01-20T08:59:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.729+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00bd10 con 0x7f6b141517f0 2024-01-20T08:59:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.729+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c014640 con 0x7f6b141517f0 2024-01-20T08:59:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.729+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc009400 con 0x7f6b141517f0 2024-01-20T08:59:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.729+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b141590e0 con 0x7f6b141517f0 2024-01-20T08:59:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.729+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b14067890 con 0x7f6b141517f0 2024-01-20T08:59:03.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.968+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6b0c0147a0 con 0x7f6b141517f0 2024-01-20T08:59:03.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.969+0000 7f6b10ff9640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6af406af20 0x7f6af406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:03.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.969+0000 7f6b137fe640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6af406af20 0x7f6af406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:03.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.969+0000 7f6b137fe640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6af406af20 0x7f6af406d3e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009aa0 tx=0x7f6b04009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:03.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.982+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f6b0c059080 con 0x7f6b141517f0 2024-01-20T08:59:03.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:03.982+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6b0c0553c0 con 0x7f6b141517f0 2024-01-20T08:59:04.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.157+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b140c9d00 con 0x7f6af406af20 2024-01-20T08:59:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.158+0000 7f6b10ff9640 1 -- 172.21.15.87:0/1938172732 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6b04011a00 con 0x7f6af406af20 2024-01-20T08:59:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 0 to deep-scrub 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6af406af20 msgr2=0x7f6af406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6af406af20 0x7f6af406d3e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009aa0 tx=0x7f6b04009190 comp rx=0 tx=0).stop 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 msgr2=0x7f6b141598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 0x7f6b141598b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00eed0 tx=0x7f6b0c00b480 comp rx=0 tx=0).stop 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 shutdown_connections 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f6af406af20 0x7f6af406d3e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b141517f0 0x7f6b141598b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b140c9520 0x7f6b14159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 --2- 172.21.15.87:0/1938172732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b140c90e0 0x7f6b141f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 >> 172.21.15.87:0/1938172732 conn(0x7f6b1406a5b0 msgr2=0x7f6b14156950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:04.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 shutdown_connections 2024-01-20T08:59:04.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.160+0000 7f6b19f10640 1 -- 172.21.15.87:0/1938172732 wait complete. 2024-01-20T08:59:04.172 INFO:tasks.ceph:Scrubbing osd.1 2024-01-20T08:59:04.172 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:04.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.279+0000 7ff07c42f640 1 Processor -- start 2024-01-20T08:59:04.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.279+0000 7ff07c42f640 1 -- start start 2024-01-20T08:59:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.280+0000 7ff07c42f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff0740bff90 0x7ff0740c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.280+0000 7ff07c42f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff074071910 con 0x7ff074156ea0 2024-01-20T08:59:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.280+0000 7ff07c42f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff074150730 con 0x7ff074150350 2024-01-20T08:59:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.280+0000 7ff07c42f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff074071b60 con 0x7ff0740bff90 2024-01-20T08:59:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.281+0000 7ff0799ca640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff0740bff90 0x7ff0740c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.281+0000 7ff07a9cc640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff074156ea0 0x7ff074160c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34184/0 (socket says 172.21.15.87:34184) 2024-01-20T08:59:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.281+0000 7ff07a9cc640 1 -- 172.21.15.87:0/3041657691 learned_addr learned my addr 172.21.15.87:0/3041657691 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.285+0000 7ff0799ca640 1 -- 172.21.15.87:0/3041657691 >> v1:172.21.15.116:6789/0 conn(0x7ff074150350 legacy=0x7ff0740cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.285+0000 7ff0799ca640 1 -- 172.21.15.87:0/3041657691 >> v1:172.21.15.87:6789/0 conn(0x7ff074156ea0 legacy=0x7ff074160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.285+0000 7ff0799ca640 1 -- 172.21.15.87:0/3041657691 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0741563c0 con 0x7ff0740bff90 2024-01-20T08:59:04.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.289+0000 7ff0799ca640 1 --2- 172.21.15.87:0/3041657691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0740bff90 0x7ff0740c57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff0700097a0 tx=0x7ff07002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11189d00899f888 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.294+0000 7ff0791c9640 1 -- 172.21.15.87:0/3041657691 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070031830 con 0x7ff0740bff90 2024-01-20T08:59:04.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.294+0000 7ff0791c9640 1 -- 172.21.15.87:0/3041657691 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070031990 con 0x7ff0740bff90 2024-01-20T08:59:04.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.294+0000 7ff0791c9640 1 -- 172.21.15.87:0/3041657691 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070031d20 con 0x7ff0740bff90 2024-01-20T08:59:04.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.295+0000 7ff07c42f640 1 -- 172.21.15.87:0/3041657691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0740bff90 msgr2=0x7ff0740c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.295+0000 7ff07c42f640 1 --2- 172.21.15.87:0/3041657691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0740bff90 0x7ff0740c57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff0700097a0 tx=0x7ff07002b8d0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.295+0000 7ff07c42f640 1 -- 172.21.15.87:0/3041657691 shutdown_connections 2024-01-20T08:59:04.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.295+0000 7ff07c42f640 1 --2- 172.21.15.87:0/3041657691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0740bff90 0x7ff0740c57c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.295+0000 7ff07c42f640 1 -- 172.21.15.87:0/3041657691 >> 172.21.15.87:0/3041657691 conn(0x7ff07406d510 msgr2=0x7ff0740663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:04.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.296+0000 7ff07c42f640 1 -- 172.21.15.87:0/3041657691 shutdown_connections 2024-01-20T08:59:04.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.296+0000 7ff07c42f640 1 -- 172.21.15.87:0/3041657691 wait complete. 2024-01-20T08:59:04.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 Processor -- start 2024-01-20T08:59:04.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 -- start start 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0740bff90 0x7ff0740c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff074150350 0x7ff074159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff074156ea0 0x7ff074148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740c3700 con 0x7ff074156ea0 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074158620 con 0x7ff074150350 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.297+0000 7ff07c42f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff074157470 con 0x7ff0740bff90 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.298+0000 7ff07a9cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff074156ea0 0x7ff074148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.299+0000 7ff07a9cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff074156ea0 0x7ff074148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60066/0 (socket says 172.21.15.87:60066) 2024-01-20T08:59:04.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.299+0000 7ff07a9cc640 1 -- 172.21.15.87:0/1224523619 learned_addr learned my addr 172.21.15.87:0/1224523619 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:04.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.302+0000 7ff07a9cc640 1 -- 172.21.15.87:0/1224523619 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0740bff90 msgr2=0x7ff0740c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.302+0000 7ff07a9cc640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0740bff90 0x7ff0740c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.302+0000 7ff07a9cc640 1 -- 172.21.15.87:0/1224523619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff074150350 msgr2=0x7ff074159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.302+0000 7ff07a9cc640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff074150350 0x7ff074159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.302+0000 7ff07a9cc640 1 -- 172.21.15.87:0/1224523619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff070009400 con 0x7ff074156ea0 2024-01-20T08:59:04.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.305+0000 7ff07a9cc640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff074156ea0 0x7ff074148b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00ca80 tx=0x7ff06c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.309+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00ac30 con 0x7ff074156ea0 2024-01-20T08:59:04.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.310+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c00ad90 con 0x7ff074156ea0 2024-01-20T08:59:04.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.310+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c012600 con 0x7ff074156ea0 2024-01-20T08:59:04.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.310+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0740caaa0 con 0x7ff074156ea0 2024-01-20T08:59:04.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.310+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff074157a10 con 0x7ff074156ea0 2024-01-20T08:59:04.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.311+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff074071e40 con 0x7ff074156ea0 2024-01-20T08:59:04.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.333+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff06c007070 con 0x7ff074156ea0 2024-01-20T08:59:04.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.333+0000 7ff06affd640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05806af20 0x7ff05806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.333+0000 7ff07a1cb640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05806af20 0x7ff05806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.334+0000 7ff07a1cb640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05806af20 0x7ff05806d3e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff064001560 tx=0x7ff064007990 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.336+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7ff06c0588b0 con 0x7ff074156ea0 2024-01-20T08:59:04.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.336+0000 7ff06affd640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7ff058074520 0x7ff0580769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.336+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 --> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff058077070 con 0x7ff058074520 2024-01-20T08:59:04.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.336+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff06c001b60 con 0x7ff074156ea0 2024-01-20T08:59:04.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.336+0000 7ff0799ca640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7ff058074520 0x7ff0580769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.337+0000 7ff0799ca640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7ff058074520 0x7ff0580769a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.337+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== osd.1 v2:172.21.15.87:6818/2572242901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff058077070 con 0x7ff058074520 2024-01-20T08:59:04.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.350+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 --> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff074156980 con 0x7ff058074520 2024-01-20T08:59:04.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff06affd640 1 -- 172.21.15.87:0/1224523619 <== osd.1 v2:172.21.15.87:6818/2572242901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff070031b00 con 0x7ff058074520 2024-01-20T08:59:04.352 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:04.352 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:04.352 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7ff058074520 msgr2=0x7ff0580769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6818/2572242901,v1:172.21.15.87:6821/2572242901] conn(0x7ff058074520 0x7ff0580769a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05806af20 msgr2=0x7ff05806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7ff05806af20 0x7ff05806d3e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff064001560 tx=0x7ff064007990 comp rx=0 tx=0).stop 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff074156ea0 msgr2=0x7ff074148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 --2- 172.21.15.87:0/1224523619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff074156ea0 0x7ff074148b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00ca80 tx=0x7ff06c00cf40 comp rx=0 tx=0).stop 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07a9cc640 1 -- 172.21.15.87:0/1224523619 reap_dead start 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 shutdown_connections 2024-01-20T08:59:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 >> 172.21.15.87:0/1224523619 conn(0x7ff07406d510 msgr2=0x7ff07414dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 shutdown_connections 2024-01-20T08:59:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.351+0000 7ff07c42f640 1 -- 172.21.15.87:0/1224523619 wait complete. 2024-01-20T08:59:04.362 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-20T08:59:04.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.462+0000 7f650d80e640 1 Processor -- start 2024-01-20T08:59:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.462+0000 7f650d80e640 1 -- start start 2024-01-20T08:59:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.463+0000 7f650d80e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6508153880 0x7f650806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.463+0000 7f650d80e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65080709e0 con 0x7f65080b1220 2024-01-20T08:59:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.463+0000 7f650d80e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650815a130 con 0x7f65080b0c60 2024-01-20T08:59:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.463+0000 7f650d80e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65081595f0 con 0x7f6508153880 2024-01-20T08:59:04.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.464+0000 7f65067fc640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f65080b1220 0x7f650815cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34186/0 (socket says 172.21.15.87:34186) 2024-01-20T08:59:04.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.464+0000 7f65067fc640 1 -- 172.21.15.87:0/2843606359 learned_addr learned my addr 172.21.15.87:0/2843606359 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:04.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.464+0000 7f6506ffd640 1 --2- 172.21.15.87:0/2843606359 >> v2:172.21.15.116:3300/0 conn(0x7f6508153880 0x7f650806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.466+0000 7f6505ffb640 1 -- 172.21.15.87:0/2843606359 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348352857 0 0) 0x7f64fc001110 con 0x7f65080b1220 2024-01-20T08:59:04.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.466+0000 7f6505ffb640 1 -- 172.21.15.87:0/2843606359 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f0003520 con 0x7f65080b1220 2024-01-20T08:59:04.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.467+0000 7f6506ffd640 1 -- 172.21.15.87:0/2843606359 >> v1:172.21.15.116:6789/0 conn(0x7f65080b0c60 legacy=0x7f650806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.467+0000 7f6506ffd640 1 -- 172.21.15.87:0/2843606359 >> v1:172.21.15.87:6789/0 conn(0x7f65080b1220 legacy=0x7f650815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.467+0000 7f6506ffd640 1 -- 172.21.15.87:0/2843606359 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6508070720 con 0x7f6508153880 2024-01-20T08:59:04.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.471+0000 7f6506ffd640 1 --2- 172.21.15.87:0/2843606359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f650806df10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009d90 tx=0x7f64f802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b318e5de02d5d112 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.475+0000 7f6505ffb640 1 -- 172.21.15.87:0/2843606359 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f802cd80 con 0x7f6508153880 2024-01-20T08:59:04.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.475+0000 7f6505ffb640 1 -- 172.21.15.87:0/2843606359 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8034440 con 0x7f6508153880 2024-01-20T08:59:04.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.475+0000 7f6505ffb640 1 -- 172.21.15.87:0/2843606359 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80347d0 con 0x7f6508153880 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 -- 172.21.15.87:0/2843606359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 msgr2=0x7f650806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 --2- 172.21.15.87:0/2843606359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f650806df10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009d90 tx=0x7f64f802bfd0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 -- 172.21.15.87:0/2843606359 shutdown_connections 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 --2- 172.21.15.87:0/2843606359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f650806df10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 -- 172.21.15.87:0/2843606359 >> 172.21.15.87:0/2843606359 conn(0x7f6508072390 msgr2=0x7f650806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 -- 172.21.15.87:0/2843606359 shutdown_connections 2024-01-20T08:59:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.476+0000 7f650d80e640 1 -- 172.21.15.87:0/2843606359 wait complete. 2024-01-20T08:59:04.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.477+0000 7f650d80e640 1 Processor -- start 2024-01-20T08:59:04.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.477+0000 7f650d80e640 1 -- start start 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f650d80e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65080b0c60 0x7f6508150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f650d80e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65080b1220 0x7f65081f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f650d80e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f65080ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f650d80e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080cbb50 con 0x7f65080b0c60 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f650d80e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65081f3c70 con 0x7f6508153880 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f650d80e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65081514b0 con 0x7f65080b1220 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.478+0000 7f6506ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65080b0c60 0x7f6508150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.479+0000 7f65077fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f65080ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.479+0000 7f65067fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65080b1220 0x7f65081f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.479+0000 7f65077fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f65080ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58146/0 (socket says 172.21.15.87:58146) 2024-01-20T08:59:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.479+0000 7f65077fe640 1 -- 172.21.15.87:0/3241767898 learned_addr learned my addr 172.21.15.87:0/3241767898 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:04.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.482+0000 7f65077fe640 1 -- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65080b1220 msgr2=0x7f65081f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.482+0000 7f65077fe640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65080b1220 0x7f65081f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.482+0000 7f65077fe640 1 -- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65080b0c60 msgr2=0x7f6508150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.482+0000 7f65077fe640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65080b0c60 0x7f6508150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.482+0000 7f65077fe640 1 -- 172.21.15.87:0/3241767898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8009a70 con 0x7f6508153880 2024-01-20T08:59:04.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.482+0000 7f6506ffd640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65080b0c60 0x7f6508150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:04.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.484+0000 7f65077fe640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f65080ca650 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f650000c3c0 tx=0x7f650000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.488+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65000052d0 con 0x7f6508153880 2024-01-20T08:59:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.488+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6500005430 con 0x7f6508153880 2024-01-20T08:59:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.488+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65081ef7b0 con 0x7f6508153880 2024-01-20T08:59:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.488+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500003810 con 0x7f6508153880 2024-01-20T08:59:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.488+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65081efa70 con 0x7f6508153880 2024-01-20T08:59:04.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.489+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650815a5a0 con 0x7f6508153880 2024-01-20T08:59:04.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.507+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6500003970 con 0x7f6508153880 2024-01-20T08:59:04.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.507+0000 7f64effff640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f64f406af20 0x7f64f406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:04.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.507+0000 7f6506ffd640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f64f406af20 0x7f64f406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:04.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.508+0000 7f6506ffd640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f64f406af20 0x7f64f406d3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64f8036000 tx=0x7f64f8030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.511+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f65000577b0 con 0x7f6508153880 2024-01-20T08:59:04.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.752+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6500053af0 con 0x7f6508153880 2024-01-20T08:59:04.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.925+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6508159ae0 con 0x7f64f406af20 2024-01-20T08:59:04.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.925+0000 7f64effff640 1 -- 172.21.15.87:0/3241767898 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f64f8034980 con 0x7f64f406af20 2024-01-20T08:59:04.926 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 1 to deep-scrub 2024-01-20T08:59:04.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f64f406af20 msgr2=0x7f64f406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f64f406af20 0x7f64f406d3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64f8036000 tx=0x7f64f8030000 comp rx=0 tx=0).stop 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 msgr2=0x7f65080ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f65080ca650 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f650000c3c0 tx=0x7f650000c880 comp rx=0 tx=0).stop 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 shutdown_connections 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f64f406af20 0x7f64f406d3e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6508153880 0x7f65080ca650 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65080b1220 0x7f65081f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 --2- 172.21.15.87:0/3241767898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65080b0c60 0x7f6508150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 >> 172.21.15.87:0/3241767898 conn(0x7f6508072390 msgr2=0x7f650814b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 shutdown_connections 2024-01-20T08:59:04.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:04.927+0000 7f650d80e640 1 -- 172.21.15.87:0/3241767898 wait complete. 2024-01-20T08:59:04.939 INFO:tasks.ceph:Scrubbing osd.2 2024-01-20T08:59:04.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:05.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.042+0000 7f5b7cd5f640 1 Processor -- start 2024-01-20T08:59:05.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.042+0000 7f5b7cd5f640 1 -- start start 2024-01-20T08:59:05.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.043+0000 7f5b7cd5f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b78153560 0x7f5b780c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.043+0000 7f5b7cd5f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5b7806a5b0 0x7f5b78159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.043+0000 7f5b7cd5f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b78072560 con 0x7f5b7806fcf0 2024-01-20T08:59:05.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.043+0000 7f5b7cd5f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b78072a90 con 0x7f5b78153560 2024-01-20T08:59:05.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.043+0000 7f5b7cd5f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b780c7bd0 con 0x7f5b7806a5b0 2024-01-20T08:59:05.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.044+0000 7f5b7659c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b78153560 0x7f5b780c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.044+0000 7f5b75d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5b7806a5b0 0x7f5b78159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.044+0000 7f5b7659c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b78153560 0x7f5b780c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60094/0 (socket says 172.21.15.87:60094) 2024-01-20T08:59:05.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.044+0000 7f5b7659c640 1 -- 172.21.15.87:0/1317280306 learned_addr learned my addr 172.21.15.87:0/1317280306 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.044+0000 7f5b76d9d640 1 --1- 172.21.15.87:0/1317280306 >> v1:172.21.15.116:6789/0 conn(0x7f5b7806fcf0 0x7f5b78159b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:42028/0 (socket says 172.21.15.87:42028) 2024-01-20T08:59:05.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.046+0000 7f5b7659c640 1 -- 172.21.15.87:0/1317280306 >> v2:172.21.15.116:3300/0 conn(0x7f5b7806a5b0 msgr2=0x7f5b78159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:05.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.046+0000 7f5b7659c640 1 --2- 172.21.15.87:0/1317280306 >> v2:172.21.15.116:3300/0 conn(0x7f5b7806a5b0 0x7f5b78159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.046+0000 7f5b7659c640 1 -- 172.21.15.87:0/1317280306 >> v1:172.21.15.116:6789/0 conn(0x7f5b7806fcf0 legacy=0x7f5b78159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.046+0000 7f5b7659c640 1 -- 172.21.15.87:0/1317280306 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b780727b0 con 0x7f5b78153560 2024-01-20T08:59:05.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.048+0000 7f5b7659c640 1 --2- 172.21.15.87:0/1317280306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78153560 0x7f5b780c7690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b780c9180 tx=0x7f5b6802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5718b2895081582f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.052+0000 7f5b7559a640 1 -- 172.21.15.87:0/1317280306 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6802cb20 con 0x7f5b78153560 2024-01-20T08:59:05.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.052+0000 7f5b7559a640 1 -- 172.21.15.87:0/1317280306 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6802cc80 con 0x7f5b78153560 2024-01-20T08:59:05.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.052+0000 7f5b7559a640 1 -- 172.21.15.87:0/1317280306 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680343f0 con 0x7f5b78153560 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.052+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1317280306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78153560 msgr2=0x7f5b780c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 --2- 172.21.15.87:0/1317280306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78153560 0x7f5b780c7690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b780c9180 tx=0x7f5b6802bf80 comp rx=0 tx=0).stop 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1317280306 shutdown_connections 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 --2- 172.21.15.87:0/1317280306 >> v2:172.21.15.116:3300/0 conn(0x7f5b7806a5b0 0x7f5b78159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 --2- 172.21.15.87:0/1317280306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78153560 0x7f5b780c7690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1317280306 >> 172.21.15.87:0/1317280306 conn(0x7f5b78071690 msgr2=0x7f5b78071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1317280306 shutdown_connections 2024-01-20T08:59:05.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.053+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1317280306 wait complete. 2024-01-20T08:59:05.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.054+0000 7f5b7cd5f640 1 Processor -- start 2024-01-20T08:59:05.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.054+0000 7f5b7cd5f640 1 -- start start 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7cd5f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b7806a5b0 0x7f5b7814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7cd5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b7806fcf0 0x7f5b781594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7cd5f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b78153560 0x7f5b781ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7cd5f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781eefb0 con 0x7f5b7806a5b0 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7cd5f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781599f0 con 0x7f5b7806fcf0 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7cd5f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b78155230 con 0x7f5b78153560 2024-01-20T08:59:05.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b7806a5b0 0x7f5b7814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b75d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b7806fcf0 0x7f5b781594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b7806a5b0 0x7f5b7814a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60100/0 (socket says 172.21.15.87:60100) 2024-01-20T08:59:05.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.055+0000 7f5b7659c640 1 -- 172.21.15.87:0/1031326319 learned_addr learned my addr 172.21.15.87:0/1031326319 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.057+0000 7f5b76d9d640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b78153560 0x7f5b781ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.059+0000 7f5b7659c640 1 -- 172.21.15.87:0/1031326319 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b78153560 msgr2=0x7f5b781ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.059+0000 7f5b7659c640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b78153560 0x7f5b781ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.059+0000 7f5b7659c640 1 -- 172.21.15.87:0/1031326319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b7806fcf0 msgr2=0x7f5b781594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.059+0000 7f5b7659c640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b7806fcf0 0x7f5b781594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.059+0000 7f5b7659c640 1 -- 172.21.15.87:0/1031326319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68009b30 con 0x7f5b7806a5b0 2024-01-20T08:59:05.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.061+0000 7f5b7659c640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b7806a5b0 0x7f5b7814a3a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b68036040 tx=0x7f5b68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.065+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68009510 con 0x7f5b7806a5b0 2024-01-20T08:59:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.065+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68007e60 con 0x7f5b7806a5b0 2024-01-20T08:59:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.065+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b78155510 con 0x7f5b7806a5b0 2024-01-20T08:59:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.065+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6803e690 con 0x7f5b7806a5b0 2024-01-20T08:59:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.065+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b78155ad0 con 0x7f5b7806a5b0 2024-01-20T08:59:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.066+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b78066aa0 con 0x7f5b7806a5b0 2024-01-20T08:59:05.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.084+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5b6803e7f0 con 0x7f5b7806a5b0 2024-01-20T08:59:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.085+0000 7f5b66ffd640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5b5806af50 0x7f5b5806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.085+0000 7f5b75d9b640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5b5806af50 0x7f5b5806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.085+0000 7f5b75d9b640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5b5806af50 0x7f5b5806d410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b7815a9a0 tx=0x7f5b6c009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.087+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f5b6807b170 con 0x7f5b7806a5b0 2024-01-20T08:59:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.087+0000 7f5b66ffd640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f5b58074550 0x7f5b580769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.087+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 --> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b580770a0 con 0x7f5b58074550 2024-01-20T08:59:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.087+0000 7f5b76d9d640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f5b58074550 0x7f5b580769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.088+0000 7f5b76d9d640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f5b58074550 0x7f5b580769d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.088+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== osd.2 v2:172.21.15.87:6808/3102513365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5b580770a0 con 0x7f5b58074550 2024-01-20T08:59:05.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.091+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b680774b0 con 0x7f5b7806a5b0 2024-01-20T08:59:05.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 --> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5b780667c0 con 0x7f5b58074550 2024-01-20T08:59:05.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b66ffd640 1 -- 172.21.15.87:0/1031326319 <== osd.2 v2:172.21.15.87:6808/3102513365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5b70007300 con 0x7f5b58074550 2024-01-20T08:59:05.103 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:05.104 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:05.104 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f5b58074550 msgr2=0x7f5b580769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6808/3102513365,v1:172.21.15.87:6809/3102513365] conn(0x7f5b58074550 0x7f5b580769d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5b5806af50 msgr2=0x7f5b5806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5b5806af50 0x7f5b5806d410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b7815a9a0 tx=0x7f5b6c009190 comp rx=0 tx=0).stop 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b7806a5b0 msgr2=0x7f5b7814a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.102+0000 7f5b7cd5f640 1 --2- 172.21.15.87:0/1031326319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b7806a5b0 0x7f5b7814a3a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b68036040 tx=0x7f5b68004a40 comp rx=0 tx=0).stop 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.103+0000 7f5b76d9d640 1 -- 172.21.15.87:0/1031326319 reap_dead start 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.103+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 shutdown_connections 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.103+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 >> 172.21.15.87:0/1031326319 conn(0x7f5b78071690 msgr2=0x7f5b780c5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.103+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 shutdown_connections 2024-01-20T08:59:05.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.103+0000 7f5b7cd5f640 1 -- 172.21.15.87:0/1031326319 wait complete. 2024-01-20T08:59:05.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-20T08:59:05.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.215+0000 7fa53f64b640 1 Processor -- start 2024-01-20T08:59:05.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.215+0000 7fa53f64b640 1 -- start start 2024-01-20T08:59:05.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.216+0000 7fa53f64b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5380c0e80 0x7fa53815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.216+0000 7fa53f64b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa5380c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.216+0000 7fa53f64b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53815a820 con 0x7fa538153ab0 2024-01-20T08:59:05.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.216+0000 7fa53f64b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53806ec50 con 0x7fa5380c0e80 2024-01-20T08:59:05.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.216+0000 7fa53f64b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c1330 con 0x7fa5380c6760 2024-01-20T08:59:05.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.217+0000 7fa53cbe6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5380c0e80 0x7fa53815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.217+0000 7fa53dbe8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa5380c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.217+0000 7fa53dbe8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa5380c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50228/0 (socket says 172.21.15.87:50228) 2024-01-20T08:59:05.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.217+0000 7fa53cbe6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5380c0e80 0x7fa53815a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58180/0 (socket says 172.21.15.87:58180) 2024-01-20T08:59:05.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.217+0000 7fa53dbe8640 1 -- 172.21.15.87:0/3429647831 learned_addr learned my addr 172.21.15.87:0/3429647831 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.219+0000 7fa53cbe6640 1 -- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 msgr2=0x7fa5380c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:05.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.219+0000 7fa53cbe6640 1 --2- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa5380c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.219+0000 7fa53cbe6640 1 -- 172.21.15.87:0/3429647831 >> v1:172.21.15.87:6789/0 conn(0x7fa538153ab0 legacy=0x7fa5380c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:05.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.219+0000 7fa53cbe6640 1 -- 172.21.15.87:0/3429647831 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53806de90 con 0x7fa5380c0e80 2024-01-20T08:59:05.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.221+0000 7fa53cbe6640 1 --2- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5380c0e80 0x7fa53815a2e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa5280097a0 tx=0x7fa52802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e8bab4d6d100e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.225+0000 7fa52ffff640 1 -- 172.21.15.87:0/3429647831 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52802f050 con 0x7fa5380c0e80 2024-01-20T08:59:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.225+0000 7fa52ffff640 1 -- 172.21.15.87:0/3429647831 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa528031700 con 0x7fa5380c0e80 2024-01-20T08:59:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.225+0000 7fa52ffff640 1 -- 172.21.15.87:0/3429647831 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528031a90 con 0x7fa5380c0e80 2024-01-20T08:59:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 -- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5380c0e80 msgr2=0x7fa53815a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 --2- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5380c0e80 0x7fa53815a2e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa5280097a0 tx=0x7fa52802b7a0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 -- 172.21.15.87:0/3429647831 shutdown_connections 2024-01-20T08:59:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 --2- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa5380c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 --2- 172.21.15.87:0/3429647831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5380c0e80 0x7fa53815a2e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 -- 172.21.15.87:0/3429647831 >> 172.21.15.87:0/3429647831 conn(0x7fa5380723c0 msgr2=0x7fa53806fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 -- 172.21.15.87:0/3429647831 shutdown_connections 2024-01-20T08:59:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.226+0000 7fa53f64b640 1 -- 172.21.15.87:0/3429647831 wait complete. 2024-01-20T08:59:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.227+0000 7fa53f64b640 1 Processor -- start 2024-01-20T08:59:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.227+0000 7fa53f64b640 1 -- start start 2024-01-20T08:59:05.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53f64b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5380c0e80 0x7fa5381ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53f64b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa538155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53f64b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa538153ab0 0x7fa5380cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53f64b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c9b60 con 0x7fa5380c0e80 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53f64b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538152420 con 0x7fa538153ab0 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53f64b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa538159650 con 0x7fa5380c6760 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53dbe8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa538153ab0 0x7fa5380cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.228+0000 7fa53cbe6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa538155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.229+0000 7fa53cbe6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa538155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:50238/0 (socket says 172.21.15.87:50238) 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.229+0000 7fa53cbe6640 1 -- 172.21.15.87:0/2724226272 learned_addr learned my addr 172.21.15.87:0/2724226272 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.229+0000 7fa53dbe8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa538153ab0 0x7fa5380cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:58190/0 (socket says 172.21.15.87:58190) 2024-01-20T08:59:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.232+0000 7fa53cbe6640 1 -- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa538153ab0 msgr2=0x7fa5380cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.232+0000 7fa53cbe6640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa538153ab0 0x7fa5380cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.232+0000 7fa53cbe6640 1 -- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5380c0e80 msgr2=0x7fa5381ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.232+0000 7fa53cbe6640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5380c0e80 0x7fa5381ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.232+0000 7fa53cbe6640 1 -- 172.21.15.87:0/2724226272 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528009400 con 0x7fa5380c6760 2024-01-20T08:59:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.235+0000 7fa53cbe6640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa538155b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098c0 tx=0x7fa5280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.238+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528033ab0 con 0x7fa5380c6760 2024-01-20T08:59:05.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.238+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa528033c30 con 0x7fa5380c6760 2024-01-20T08:59:05.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.238+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52803e480 con 0x7fa5380c6760 2024-01-20T08:59:05.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.238+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5380c9db0 con 0x7fa5380c6760 2024-01-20T08:59:05.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.239+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5380ca370 con 0x7fa5380c6760 2024-01-20T08:59:05.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.240+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa538074290 con 0x7fa5380c6760 2024-01-20T08:59:05.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.482+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa52803a040 con 0x7fa5380c6760 2024-01-20T08:59:05.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.482+0000 7fa52dffb640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa51c06af50 0x7fa51c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.482+0000 7fa53d3e7640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa51c06af50 0x7fa51c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.483+0000 7fa53d3e7640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa51c06af50 0x7fa51c06d410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa534009650 tx=0x7fa534006ba0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.488+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fa52807ab00 con 0x7fa5380c6760 2024-01-20T08:59:05.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.500+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa528076740 con 0x7fa5380c6760 2024-01-20T08:59:05.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.678+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa53806e730 con 0x7fa51c06af50 2024-01-20T08:59:05.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.678+0000 7fa52dffb640 1 -- 172.21.15.87:0/2724226272 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa534013430 con 0x7fa51c06af50 2024-01-20T08:59:05.679 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 2 to deep-scrub 2024-01-20T08:59:05.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa51c06af50 msgr2=0x7fa51c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa51c06af50 0x7fa51c06d410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa534009650 tx=0x7fa534006ba0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 msgr2=0x7fa538155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa538155b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098c0 tx=0x7fa5280046c0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 shutdown_connections 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fa51c06af50 0x7fa51c06d410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa538153ab0 0x7fa5380cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5380c6760 0x7fa538155b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 --2- 172.21.15.87:0/2724226272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5380c0e80 0x7fa5381ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 >> 172.21.15.87:0/2724226272 conn(0x7fa5380723c0 msgr2=0x7fa53806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 shutdown_connections 2024-01-20T08:59:05.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.680+0000 7fa53f64b640 1 -- 172.21.15.87:0/2724226272 wait complete. 2024-01-20T08:59:05.692 INFO:tasks.ceph:Scrubbing osd.3 2024-01-20T08:59:05.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:05.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.796+0000 7fca4fed5640 1 Processor -- start 2024-01-20T08:59:05.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.797+0000 7fca4fed5640 1 -- start start 2024-01-20T08:59:05.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.797+0000 7fca4fed5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca480c9ec0 0x7fca480cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.797+0000 7fca4fed5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fca480ccc80 0x7fca480cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.797+0000 7fca4fed5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca480cbfb0 con 0x7fca480ce320 2024-01-20T08:59:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.797+0000 7fca4fed5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca480cdd10 con 0x7fca480c9ec0 2024-01-20T08:59:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.797+0000 7fca4fed5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca480cdff0 con 0x7fca480ccc80 2024-01-20T08:59:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.798+0000 7fca4dc71640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca480c9ec0 0x7fca480cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.798+0000 7fca4d470640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fca480ccc80 0x7fca480cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.798+0000 7fca4d470640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fca480ccc80 0x7fca480cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41380/0 (socket says 172.21.15.87:41380) 2024-01-20T08:59:05.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.798+0000 7fca4d470640 1 -- 172.21.15.87:0/505584827 learned_addr learned my addr 172.21.15.87:0/505584827 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.800+0000 7fca4d470640 1 -- 172.21.15.87:0/505584827 >> v2:172.21.15.87:3300/0 conn(0x7fca480c9ec0 msgr2=0x7fca480cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:05.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.800+0000 7fca4d470640 1 --2- 172.21.15.87:0/505584827 >> v2:172.21.15.87:3300/0 conn(0x7fca480c9ec0 0x7fca480cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.800+0000 7fca4d470640 1 -- 172.21.15.87:0/505584827 >> v1:172.21.15.87:6789/0 conn(0x7fca480ce320 legacy=0x7fca481597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.800+0000 7fca4d470640 1 -- 172.21.15.87:0/505584827 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480cb680 con 0x7fca480ccc80 2024-01-20T08:59:05.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.802+0000 7fca4d470640 1 --2- 172.21.15.87:0/505584827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ccc80 0x7fca480cd0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca48157420 tx=0x7fca3802ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4df56abf1ecce9f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4cc6f640 1 -- 172.21.15.87:0/505584827 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3802f850 con 0x7fca480ccc80 2024-01-20T08:59:05.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4cc6f640 1 -- 172.21.15.87:0/505584827 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3802f9b0 con 0x7fca480ccc80 2024-01-20T08:59:05.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4cc6f640 1 -- 172.21.15.87:0/505584827 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3802fd40 con 0x7fca480ccc80 2024-01-20T08:59:05.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 -- 172.21.15.87:0/505584827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ccc80 msgr2=0x7fca480cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 --2- 172.21.15.87:0/505584827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ccc80 0x7fca480cd0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca48157420 tx=0x7fca3802ba30 comp rx=0 tx=0).stop 2024-01-20T08:59:05.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 -- 172.21.15.87:0/505584827 shutdown_connections 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 --2- 172.21.15.87:0/505584827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ccc80 0x7fca480cd0a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 --2- 172.21.15.87:0/505584827 >> v2:172.21.15.87:3300/0 conn(0x7fca480c9ec0 0x7fca480cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 -- 172.21.15.87:0/505584827 >> 172.21.15.87:0/505584827 conn(0x7fca480b09b0 msgr2=0x7fca480b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 -- 172.21.15.87:0/505584827 shutdown_connections 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.806+0000 7fca4fed5640 1 -- 172.21.15.87:0/505584827 wait complete. 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 Processor -- start 2024-01-20T08:59:05.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 -- start start 2024-01-20T08:59:05.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca480c9ec0 0x7fca4814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fca480ccc80 0x7fca4815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ce320 0x7fca481507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4814d7d0 con 0x7fca480c9ec0 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4814ebb0 con 0x7fca480ce320 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.808+0000 7fca4fed5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca481ef390 con 0x7fca480ccc80 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.809+0000 7fca4dc71640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca480c9ec0 0x7fca4814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.809+0000 7fca4dc71640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca480c9ec0 0x7fca4814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51462/0 (socket says 172.21.15.87:51462) 2024-01-20T08:59:05.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.809+0000 7fca4dc71640 1 -- 172.21.15.87:0/2795609755 learned_addr learned my addr 172.21.15.87:0/2795609755 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.809+0000 7fca4d470640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fca480ccc80 0x7fca4815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.809+0000 7fca4e472640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ce320 0x7fca481507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.812+0000 7fca4dc71640 1 -- 172.21.15.87:0/2795609755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fca480ccc80 msgr2=0x7fca4815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.812+0000 7fca4dc71640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fca480ccc80 0x7fca4815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.812+0000 7fca4dc71640 1 -- 172.21.15.87:0/2795609755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ce320 msgr2=0x7fca481507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.812+0000 7fca4dc71640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca480ce320 0x7fca481507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.812+0000 7fca4dc71640 1 -- 172.21.15.87:0/2795609755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380094d0 con 0x7fca480c9ec0 2024-01-20T08:59:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.815+0000 7fca4dc71640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca480c9ec0 0x7fca4814fce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca4400e9b0 tx=0x7fca4400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.818+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44009670 con 0x7fca480c9ec0 2024-01-20T08:59:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.818+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4400c9a0 con 0x7fca480c9ec0 2024-01-20T08:59:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.818+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4814eda0 con 0x7fca480c9ec0 2024-01-20T08:59:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.818+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4400cd50 con 0x7fca480c9ec0 2024-01-20T08:59:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.819+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca481ef930 con 0x7fca480c9ec0 2024-01-20T08:59:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.819+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca480cbc40 con 0x7fca480c9ec0 2024-01-20T08:59:05.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.839+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fca44013470 con 0x7fca480c9ec0 2024-01-20T08:59:05.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.839+0000 7fca3e7fc640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fca2c06af50 0x7fca2c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.839+0000 7fca4d470640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fca2c06af50 0x7fca2c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.840+0000 7fca4d470640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fca2c06af50 0x7fca2c06d410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca38036000 tx=0x7fca38008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.843+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fca44058770 con 0x7fca480c9ec0 2024-01-20T08:59:05.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.843+0000 7fca3e7fc640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7fca2c0745d0 0x7fca2c076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.843+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 --> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2c077120 con 0x7fca2c0745d0 2024-01-20T08:59:05.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.843+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca44054ab0 con 0x7fca480c9ec0 2024-01-20T08:59:05.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.843+0000 7fca4e472640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7fca2c0745d0 0x7fca2c076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.844+0000 7fca4e472640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7fca2c0745d0 0x7fca2c076a50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.844+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== osd.3 v2:172.21.15.87:6816/973971716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca2c077120 con 0x7fca2c0745d0 2024-01-20T08:59:05.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.857+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 --> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fca48074290 con 0x7fca2c0745d0 2024-01-20T08:59:05.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.857+0000 7fca3e7fc640 1 -- 172.21.15.87:0/2795609755 <== osd.3 v2:172.21.15.87:6816/973971716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fca48074290 con 0x7fca2c0745d0 2024-01-20T08:59:05.859 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:05.859 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7fca2c0745d0 msgr2=0x7fca2c076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6816/973971716,v1:172.21.15.87:6817/973971716] conn(0x7fca2c0745d0 0x7fca2c076a50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fca2c06af50 msgr2=0x7fca2c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fca2c06af50 0x7fca2c06d410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca38036000 tx=0x7fca38008030 comp rx=0 tx=0).stop 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca480c9ec0 msgr2=0x7fca4814fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 --2- 172.21.15.87:0/2795609755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca480c9ec0 0x7fca4814fce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca4400e9b0 tx=0x7fca4400ee70 comp rx=0 tx=0).stop 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4e472640 1 -- 172.21.15.87:0/2795609755 reap_dead start 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 shutdown_connections 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 >> 172.21.15.87:0/2795609755 conn(0x7fca480b09b0 msgr2=0x7fca480c8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 shutdown_connections 2024-01-20T08:59:05.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.858+0000 7fca4fed5640 1 -- 172.21.15.87:0/2795609755 wait complete. 2024-01-20T08:59:05.870 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-20T08:59:05.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.969+0000 7fda8b378640 1 Processor -- start 2024-01-20T08:59:05.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.969+0000 7fda8b378640 1 -- start start 2024-01-20T08:59:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda8b378640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda8406ef60 0x7fda8406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda8b378640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda84159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda8b378640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8415cb70 con 0x7fda84153800 2024-01-20T08:59:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda8b378640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda840c2730 con 0x7fda8406ef60 2024-01-20T08:59:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda8b378640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda840c9430 con 0x7fda84159710 2024-01-20T08:59:05.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda89114640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fda84153800 0x7fda840c99d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34780/0 (socket says 172.21.15.87:34780) 2024-01-20T08:59:05.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda89114640 1 -- 172.21.15.87:0/3083460763 learned_addr learned my addr 172.21.15.87:0/3083460763 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:05.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.970+0000 7fda89915640 1 --2- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda84159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.971+0000 7fda88913640 1 --2- 172.21.15.87:0/3083460763 >> v2:172.21.15.116:3300/0 conn(0x7fda8406ef60 0x7fda8406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.973+0000 7fda77fff640 1 -- 172.21.15.87:0/3083460763 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534378630 0 0) 0x7fda8415cb70 con 0x7fda84153800 2024-01-20T08:59:05.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.973+0000 7fda77fff640 1 -- 172.21.15.87:0/3083460763 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda70003520 con 0x7fda84153800 2024-01-20T08:59:05.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.973+0000 7fda88913640 1 -- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 msgr2=0x7fda84159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:05.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.973+0000 7fda88913640 1 --2- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda84159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.973+0000 7fda88913640 1 -- 172.21.15.87:0/3083460763 >> v1:172.21.15.87:6789/0 conn(0x7fda84153800 legacy=0x7fda840c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.973+0000 7fda88913640 1 -- 172.21.15.87:0/3083460763 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8415d9d0 con 0x7fda8406ef60 2024-01-20T08:59:05.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.975+0000 7fda88913640 1 --2- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8406f380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda800097a0 tx=0x7fda8002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8679753b0643291 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.978+0000 7fda77fff640 1 -- 172.21.15.87:0/3083460763 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80031830 con 0x7fda8406ef60 2024-01-20T08:59:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.978+0000 7fda77fff640 1 -- 172.21.15.87:0/3083460763 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80031990 con 0x7fda8406ef60 2024-01-20T08:59:05.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.978+0000 7fda77fff640 1 -- 172.21.15.87:0/3083460763 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80031d20 con 0x7fda8406ef60 2024-01-20T08:59:05.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 -- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 msgr2=0x7fda8406f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 --2- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8406f380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda800097a0 tx=0x7fda8002b8d0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 -- 172.21.15.87:0/3083460763 shutdown_connections 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 --2- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda84159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 --2- 172.21.15.87:0/3083460763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8406f380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 -- 172.21.15.87:0/3083460763 >> 172.21.15.87:0/3083460763 conn(0x7fda84070de0 msgr2=0x7fda840711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 -- 172.21.15.87:0/3083460763 shutdown_connections 2024-01-20T08:59:05.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.979+0000 7fda8b378640 1 -- 172.21.15.87:0/3083460763 wait complete. 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.980+0000 7fda8b378640 1 Processor -- start 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.980+0000 7fda8b378640 1 -- start start 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda8b378640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda8b378640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda84153800 0x7fda841519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda8b378640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda841f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda8b378640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841f0180 con 0x7fda84153800 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda8b378640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8415bc60 con 0x7fda8406ef60 2024-01-20T08:59:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda8b378640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda841f2c90 con 0x7fda84159710 2024-01-20T08:59:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.981+0000 7fda88913640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda84153800 0x7fda841519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.982+0000 7fda89915640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda841f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.982+0000 7fda89114640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.982+0000 7fda89915640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda841f0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:52852/0 (socket says 172.21.15.87:52852) 2024-01-20T08:59:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.982+0000 7fda89114640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8415c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41414/0 (socket says 172.21.15.87:41414) 2024-01-20T08:59:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.982+0000 7fda89915640 1 -- 172.21.15.87:0/2604195091 learned_addr learned my addr 172.21.15.87:0/2604195091 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:05.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda89114640 1 -- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 msgr2=0x7fda841f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda89114640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda841f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda89114640 1 -- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda84153800 msgr2=0x7fda841519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:05.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda89114640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda84153800 0x7fda841519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:05.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda89114640 1 -- 172.21.15.87:0/2604195091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80009400 con 0x7fda8406ef60 2024-01-20T08:59:05.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda88913640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda84153800 0x7fda841519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:05.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.985+0000 7fda89915640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda841f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.987+0000 7fda89114640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8415c6c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00cae0 tx=0x7fda7c00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:05.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.991+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00ac40 con 0x7fda8406ef60 2024-01-20T08:59:05.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.991+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c00ada0 con 0x7fda8406ef60 2024-01-20T08:59:05.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.991+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c012620 con 0x7fda8406ef60 2024-01-20T08:59:05.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.991+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda841f2f70 con 0x7fda8406ef60 2024-01-20T08:59:05.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.991+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda84152390 con 0x7fda8406ef60 2024-01-20T08:59:05.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:05.992+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda841f02f0 con 0x7fda8406ef60 2024-01-20T08:59:06.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.010+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fda7c007070 con 0x7fda8406ef60 2024-01-20T08:59:06.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.010+0000 7fda75ffb640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fda6806af50 0x7fda6806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.010+0000 7fda88913640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fda6806af50 0x7fda6806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.011+0000 7fda88913640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fda6806af50 0x7fda6806d410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda80036000 tx=0x7fda8003c040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.014+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fda7c0585f0 con 0x7fda8406ef60 2024-01-20T08:59:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.266+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fda7c054930 con 0x7fda8406ef60 2024-01-20T08:59:06.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.436+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fda8415d380 con 0x7fda6806af50 2024-01-20T08:59:06.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.437+0000 7fda75ffb640 1 -- 172.21.15.87:0/2604195091 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fda80031b80 con 0x7fda6806af50 2024-01-20T08:59:06.438 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 3 to deep-scrub 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fda6806af50 msgr2=0x7fda6806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fda6806af50 0x7fda6806d410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda80036000 tx=0x7fda8003c040 comp rx=0 tx=0).stop 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 msgr2=0x7fda8415c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8415c6c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00cae0 tx=0x7fda7c00a060 comp rx=0 tx=0).stop 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 shutdown_connections 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fda6806af50 0x7fda6806d410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda84159710 0x7fda841f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda84153800 0x7fda841519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 --2- 172.21.15.87:0/2604195091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda8406ef60 0x7fda8415c6c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 >> 172.21.15.87:0/2604195091 conn(0x7fda84070de0 msgr2=0x7fda8406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 shutdown_connections 2024-01-20T08:59:06.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.439+0000 7fda8b378640 1 -- 172.21.15.87:0/2604195091 wait complete. 2024-01-20T08:59:06.451 INFO:tasks.ceph:Scrubbing osd.4 2024-01-20T08:59:06.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:06.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.563+0000 7f9ff9578640 1 Processor -- start 2024-01-20T08:59:06.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.563+0000 7f9ff9578640 1 -- start start 2024-01-20T08:59:06.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.564+0000 7f9ff9578640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ff40c9ec0 0x7f9ff40cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.564+0000 7f9ff9578640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40cbfb0 con 0x7f9ff40ccc80 2024-01-20T08:59:06.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.564+0000 7f9ff9578640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40cdd10 con 0x7f9ff40cd0a0 2024-01-20T08:59:06.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.564+0000 7f9ff9578640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40cdff0 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.564+0000 7f9ff2ffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ff40c9ec0 0x7f9ff40cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.565+0000 7f9febfff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9ff40ccc80 0x7f9ff40ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34786/0 (socket says 172.21.15.87:34786) 2024-01-20T08:59:06.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.565+0000 7f9febfff640 1 -- 172.21.15.87:0/1827445707 learned_addr learned my addr 172.21.15.87:0/1827445707 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:06.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.568+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/1827445707 >> v1:172.21.15.116:6789/0 conn(0x7f9ff40cd0a0 legacy=0x7f9ff41597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.568+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/1827445707 >> v1:172.21.15.87:6789/0 conn(0x7f9ff40ccc80 legacy=0x7f9ff40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.568+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/1827445707 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40cb680 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.571+0000 7f9ff2ffd640 1 --2- 172.21.15.87:0/1827445707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff40cd660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40097a0 tx=0x7f9fe402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78361e32715343a server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.585+0000 7f9ff27fc640 1 -- 172.21.15.87:0/1827445707 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe402f810 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.585+0000 7f9ff27fc640 1 -- 172.21.15.87:0/1827445707 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe402f970 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.585+0000 7f9ff27fc640 1 -- 172.21.15.87:0/1827445707 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe402fd00 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 -- 172.21.15.87:0/1827445707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40c9ec0 msgr2=0x7f9ff40cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 --2- 172.21.15.87:0/1827445707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff40cd660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40097a0 tx=0x7f9fe402b9f0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 -- 172.21.15.87:0/1827445707 shutdown_connections 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 --2- 172.21.15.87:0/1827445707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff40cd660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 -- 172.21.15.87:0/1827445707 >> 172.21.15.87:0/1827445707 conn(0x7f9ff40b09b0 msgr2=0x7f9ff40b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 -- 172.21.15.87:0/1827445707 shutdown_connections 2024-01-20T08:59:06.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.586+0000 7f9ff9578640 1 -- 172.21.15.87:0/1827445707 wait complete. 2024-01-20T08:59:06.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.587+0000 7f9ff9578640 1 Processor -- start 2024-01-20T08:59:06.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 -- start start 2024-01-20T08:59:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff41f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40ccc80 0x7f9ff41578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ff40cd0a0 0x7f9ff41ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41efd30 con 0x7f9ff40ccc80 2024-01-20T08:59:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41f3450 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.588+0000 7f9ff9578640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41e5000 con 0x7f9ff40cd0a0 2024-01-20T08:59:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.589+0000 7f9ff2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff41f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.589+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ff40cd0a0 0x7f9ff41ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.589+0000 7f9ff2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff41f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41420/0 (socket says 172.21.15.87:41420) 2024-01-20T08:59:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.589+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/2664633617 learned_addr learned my addr 172.21.15.87:0/2664633617 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.593+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ff40cd0a0 msgr2=0x7f9ff41ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.593+0000 7f9ff2ffd640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ff40cd0a0 0x7f9ff41ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.593+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40ccc80 msgr2=0x7f9ff41578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.593+0000 7f9ff2ffd640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ff40ccc80 0x7f9ff41578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.593+0000 7f9ff2ffd640 1 -- 172.21.15.87:0/2664633617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4009400 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.597+0000 7f9ff2ffd640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff41f38e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fe402c000 tx=0x7f9fe40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.660+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe403b040 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.660+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe4037a50 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.660+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4037e00 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.660+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff41e52e0 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.660+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff41f30e0 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.661+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ff40cbc40 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.683+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9fe4042460 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.683+0000 7f9feb7fe640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fd406af20 0x7f9fd406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.684+0000 7f9febfff640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fd406af20 0x7f9fd406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.684+0000 7f9febfff640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fd406af20 0x7f9fd406d3e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc000f60 tx=0x7f9fdc007a80 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.686+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9fe407b4b0 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.686+0000 7f9feb7fe640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9fd4074520 0x7f9fd40769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.686+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 --> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fd4077070 con 0x7f9fd4074520 2024-01-20T08:59:06.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.686+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe40777f0 con 0x7f9ff40c9ec0 2024-01-20T08:59:06.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.686+0000 7f9ff37fe640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9fd4074520 0x7f9fd40769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.687+0000 7f9ff37fe640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9fd4074520 0x7f9fd40769a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.687+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== osd.4 v2:172.21.15.116:6808/2305241895 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9fd4077070 con 0x7f9fd4074520 2024-01-20T08:59:06.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.701+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 --> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9ff41f3590 con 0x7f9fd4074520 2024-01-20T08:59:06.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9feb7fe640 1 -- 172.21.15.87:0/2664633617 <== osd.4 v2:172.21.15.116:6808/2305241895 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9fec001aa0 con 0x7f9fd4074520 2024-01-20T08:59:06.703 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:06.703 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:06.703 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:06.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9fd4074520 msgr2=0x7f9fd40769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:6808/2305241895,v1:172.21.15.116:6809/2305241895] conn(0x7f9fd4074520 0x7f9fd40769a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fd406af20 msgr2=0x7f9fd406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fd406af20 0x7f9fd406d3e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc000f60 tx=0x7f9fdc007a80 comp rx=0 tx=0).stop 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ff40c9ec0 msgr2=0x7f9ff41f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 --2- 172.21.15.87:0/2664633617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ff40c9ec0 0x7f9ff41f38e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fe402c000 tx=0x7f9fe40046c0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff37fe640 1 -- 172.21.15.87:0/2664633617 reap_dead start 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 shutdown_connections 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 >> 172.21.15.87:0/2664633617 conn(0x7f9ff40b09b0 msgr2=0x7f9ff40c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 shutdown_connections 2024-01-20T08:59:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.702+0000 7f9ff9578640 1 -- 172.21.15.87:0/2664633617 wait complete. 2024-01-20T08:59:06.714 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-20T08:59:06.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.816+0000 7f33506da640 1 Processor -- start 2024-01-20T08:59:06.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.816+0000 7f33506da640 1 -- start start 2024-01-20T08:59:06.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.817+0000 7f33506da640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3348153af0 0x7f33480c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.817+0000 7f33506da640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f334806c390 0x7f334806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.817+0000 7f33506da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334815cd00 0x7f334815d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.817+0000 7f33506da640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480c3e10 con 0x7f3348153af0 2024-01-20T08:59:06.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.817+0000 7f33506da640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334815a140 con 0x7f334806c390 2024-01-20T08:59:06.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.817+0000 7f33506da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334806bf00 con 0x7f334815cd00 2024-01-20T08:59:06.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.818+0000 7f334e476640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3348153af0 0x7f33480c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.818+0000 7f334ec77640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334815cd00 0x7f334815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.818+0000 7f334dc75640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f334806c390 0x7f334806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.818+0000 7f334e476640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3348153af0 0x7f33480c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51504/0 (socket says 172.21.15.87:51504) 2024-01-20T08:59:06.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.818+0000 7f334e476640 1 -- 172.21.15.87:0/1826596323 learned_addr learned my addr 172.21.15.87:0/1826596323 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.821+0000 7f334e476640 1 -- 172.21.15.87:0/1826596323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334815cd00 msgr2=0x7f334815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.821+0000 7f334e476640 1 --2- 172.21.15.87:0/1826596323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334815cd00 0x7f334815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.821+0000 7f334e476640 1 -- 172.21.15.87:0/1826596323 >> v2:172.21.15.116:3300/0 conn(0x7f334806c390 msgr2=0x7f334806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.821+0000 7f334e476640 1 --2- 172.21.15.87:0/1826596323 >> v2:172.21.15.116:3300/0 conn(0x7f334806c390 0x7f334806c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.821+0000 7f334e476640 1 -- 172.21.15.87:0/1826596323 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480cb140 con 0x7f3348153af0 2024-01-20T08:59:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.821+0000 7f334dc75640 1 --2- 172.21.15.87:0/1826596323 >> v2:172.21.15.116:3300/0 conn(0x7f334806c390 0x7f334806c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:06.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.822+0000 7f334e476640 1 --2- 172.21.15.87:0/1826596323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3348153af0 0x7f33480c98d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f334806d380 tx=0x7f333802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba3e3e47aff8435 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.826+0000 7f334d474640 1 -- 172.21.15.87:0/1826596323 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333802cb00 con 0x7f3348153af0 2024-01-20T08:59:06.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.826+0000 7f334d474640 1 -- 172.21.15.87:0/1826596323 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333802cc60 con 0x7f3348153af0 2024-01-20T08:59:06.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.826+0000 7f334d474640 1 -- 172.21.15.87:0/1826596323 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380343f0 con 0x7f3348153af0 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 -- 172.21.15.87:0/1826596323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3348153af0 msgr2=0x7f33480c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 --2- 172.21.15.87:0/1826596323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3348153af0 0x7f33480c98d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f334806d380 tx=0x7f333802bf60 comp rx=0 tx=0).stop 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 -- 172.21.15.87:0/1826596323 shutdown_connections 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 --2- 172.21.15.87:0/1826596323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334815cd00 0x7f334815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 --2- 172.21.15.87:0/1826596323 >> v2:172.21.15.116:3300/0 conn(0x7f334806c390 0x7f334806c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 --2- 172.21.15.87:0/1826596323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3348153af0 0x7f33480c98d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 -- 172.21.15.87:0/1826596323 >> 172.21.15.87:0/1826596323 conn(0x7f334808b620 msgr2=0x7f3348071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 -- 172.21.15.87:0/1826596323 shutdown_connections 2024-01-20T08:59:06.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.827+0000 7f33506da640 1 -- 172.21.15.87:0/1826596323 wait complete. 2024-01-20T08:59:06.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.828+0000 7f33506da640 1 Processor -- start 2024-01-20T08:59:06.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.828+0000 7f33506da640 1 -- start start 2024-01-20T08:59:06.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f33506da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f334806c390 0x7f33480ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f33506da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 0x7f334815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f33506da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f334815cd00 0x7f3348143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:06.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f33506da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33481f35f0 con 0x7f334815cd00 2024-01-20T08:59:06.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f33506da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334814ec80 con 0x7f334806c390 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f33506da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3348148190 con 0x7f3348153af0 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f334ec77640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f334815cd00 0x7f3348143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f334dc75640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 0x7f334815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f334e476640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f334806c390 0x7f33480ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.829+0000 7f334dc75640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 0x7f334815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:52872/0 (socket says 172.21.15.87:52872) 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.830+0000 7f334ec77640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f334815cd00 0x7f3348143830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51508/0 (socket says 172.21.15.87:51508) 2024-01-20T08:59:06.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.830+0000 7f334dc75640 1 -- 172.21.15.87:0/1394720487 learned_addr learned my addr 172.21.15.87:0/1394720487 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:06.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.833+0000 7f334dc75640 1 -- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f334806c390 msgr2=0x7f33480ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.833+0000 7f334dc75640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f334806c390 0x7f33480ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.833+0000 7f334dc75640 1 -- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f334815cd00 msgr2=0x7f3348143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:06.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.833+0000 7f334dc75640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f334815cd00 0x7f3348143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:06.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.833+0000 7f334dc75640 1 -- 172.21.15.87:0/1394720487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0093a0 con 0x7f3348153af0 2024-01-20T08:59:06.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.835+0000 7f334dc75640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 0x7f334815bb90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f333c00e8d0 tx=0x7f333c00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:06.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.839+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c009630 con 0x7f3348153af0 2024-01-20T08:59:06.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.839+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c005990 con 0x7f3348153af0 2024-01-20T08:59:06.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.839+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338009b30 con 0x7f3348153af0 2024-01-20T08:59:06.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.839+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c005d20 con 0x7f3348153af0 2024-01-20T08:59:06.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.839+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33481f2b10 con 0x7f3348153af0 2024-01-20T08:59:06.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:06.840+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3348074290 con 0x7f3348153af0 2024-01-20T08:59:07.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.079+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f333c00c8e0 con 0x7f3348153af0 2024-01-20T08:59:07.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.080+0000 7f3346ffd640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f333406af50 0x7f333406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.080+0000 7f334e476640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f333406af50 0x7f333406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.080+0000 7f334e476640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f333406af50 0x7f333406d410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3338035000 tx=0x7f3338030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.083+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f333c058980 con 0x7f3348153af0 2024-01-20T08:59:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.094+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f333c054cc0 con 0x7f3348153af0 2024-01-20T08:59:07.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.263+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f334806bb90 con 0x7f333406af50 2024-01-20T08:59:07.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.264+0000 7f3346ffd640 1 -- 172.21.15.87:0/1394720487 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f333802ca90 con 0x7f333406af50 2024-01-20T08:59:07.266 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 4 to deep-scrub 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f333406af50 msgr2=0x7f333406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f333406af50 0x7f333406d410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3338035000 tx=0x7f3338030000 comp rx=0 tx=0).stop 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 msgr2=0x7f334815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 0x7f334815bb90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f333c00e8d0 tx=0x7f333c00ed90 comp rx=0 tx=0).stop 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 shutdown_connections 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f333406af50 0x7f333406d410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f334815cd00 0x7f3348143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3348153af0 0x7f334815bb90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 --2- 172.21.15.87:0/1394720487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f334806c390 0x7f33480ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 >> 172.21.15.87:0/1394720487 conn(0x7f334808b620 msgr2=0x7f33481599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 shutdown_connections 2024-01-20T08:59:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.266+0000 7f33506da640 1 -- 172.21.15.87:0/1394720487 wait complete. 2024-01-20T08:59:07.278 INFO:tasks.ceph:Scrubbing osd.5 2024-01-20T08:59:07.278 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:07.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.380+0000 7f4618bb7640 1 Processor -- start 2024-01-20T08:59:07.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.380+0000 7f4618bb7640 1 -- start start 2024-01-20T08:59:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.381+0000 7f4618bb7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46140cc090 0x7f461406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.381+0000 7f4618bb7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46140cc470 0x7f46140c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.381+0000 7f4618bb7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461406fac0 con 0x7f4614159770 2024-01-20T08:59:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.381+0000 7f4618bb7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461415d2c0 con 0x7f46140cc090 2024-01-20T08:59:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.381+0000 7f4618bb7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46140c6e10 con 0x7f46140cc470 2024-01-20T08:59:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.382+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46140cc470 0x7f46140c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.382+0000 7f4611d9b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4614159770 0x7f461415d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34788/0 (socket says 172.21.15.87:34788) 2024-01-20T08:59:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.382+0000 7f4611d9b640 1 -- 172.21.15.87:0/4245382609 learned_addr learned my addr 172.21.15.87:0/4245382609 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.382+0000 7f461259c640 1 --2- 172.21.15.87:0/4245382609 >> v2:172.21.15.116:3300/0 conn(0x7f46140cc090 0x7f461406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.384+0000 7f461159a640 1 -- 172.21.15.87:0/4245382609 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226230727 0 0) 0x7f461406fac0 con 0x7f4614159770 2024-01-20T08:59:07.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.384+0000 7f461159a640 1 -- 172.21.15.87:0/4245382609 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45fc003520 con 0x7f4614159770 2024-01-20T08:59:07.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.384+0000 7f4612d9d640 1 -- 172.21.15.87:0/4245382609 >> v2:172.21.15.116:3300/0 conn(0x7f46140cc090 msgr2=0x7f461406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.384+0000 7f4612d9d640 1 --2- 172.21.15.87:0/4245382609 >> v2:172.21.15.116:3300/0 conn(0x7f46140cc090 0x7f461406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.384+0000 7f4612d9d640 1 -- 172.21.15.87:0/4245382609 >> v1:172.21.15.87:6789/0 conn(0x7f4614159770 legacy=0x7f461415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.384+0000 7f4612d9d640 1 -- 172.21.15.87:0/4245382609 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461406fd60 con 0x7f46140cc470 2024-01-20T08:59:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.386+0000 7f4612d9d640 1 --2- 172.21.15.87:0/4245382609 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46140cc470 0x7f46140c68d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097a0 tx=0x7f460c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18c70e73062a1ad server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f461159a640 1 -- 172.21.15.87:0/4245382609 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c030610 con 0x7f46140cc470 2024-01-20T08:59:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f461159a640 1 -- 172.21.15.87:0/4245382609 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c030770 con 0x7f46140cc470 2024-01-20T08:59:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f461159a640 1 -- 172.21.15.87:0/4245382609 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c030b00 con 0x7f46140cc470 2024-01-20T08:59:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 -- 172.21.15.87:0/4245382609 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46140cc470 msgr2=0x7f46140c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 --2- 172.21.15.87:0/4245382609 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46140cc470 0x7f46140c68d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097a0 tx=0x7f460c02b960 comp rx=0 tx=0).stop 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 -- 172.21.15.87:0/4245382609 shutdown_connections 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 --2- 172.21.15.87:0/4245382609 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46140cc470 0x7f46140c68d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 --2- 172.21.15.87:0/4245382609 >> v2:172.21.15.116:3300/0 conn(0x7f46140cc090 0x7f461406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 -- 172.21.15.87:0/4245382609 >> 172.21.15.87:0/4245382609 conn(0x7f4614071ff0 msgr2=0x7f4614072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 -- 172.21.15.87:0/4245382609 shutdown_connections 2024-01-20T08:59:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.390+0000 7f4618bb7640 1 -- 172.21.15.87:0/4245382609 wait complete. 2024-01-20T08:59:07.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 Processor -- start 2024-01-20T08:59:07.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 -- start start 2024-01-20T08:59:07.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46140cc090 0x7f461414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46140cc470 0x7f46140c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4614159770 0x7f4614144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614156f70 con 0x7f46140cc090 2024-01-20T08:59:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461415a930 con 0x7f46140cc470 2024-01-20T08:59:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.392+0000 7f4618bb7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461415abd0 con 0x7f4614159770 2024-01-20T08:59:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.393+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4614159770 0x7f4614144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.393+0000 7f461259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46140cc090 0x7f461414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.393+0000 7f4611d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46140cc470 0x7f46140c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.393+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4614159770 0x7f4614144c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:52890/0 (socket says 172.21.15.87:52890) 2024-01-20T08:59:07.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.393+0000 7f4612d9d640 1 -- 172.21.15.87:0/3968162820 learned_addr learned my addr 172.21.15.87:0/3968162820 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:07.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.397+0000 7f4612d9d640 1 -- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46140cc470 msgr2=0x7f46140c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.397+0000 7f4612d9d640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46140cc470 0x7f46140c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.397+0000 7f4612d9d640 1 -- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46140cc090 msgr2=0x7f461414de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.397+0000 7f4612d9d640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46140cc090 0x7f461414de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.397+0000 7f4612d9d640 1 -- 172.21.15.87:0/3968162820 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c009400 con 0x7f4614159770 2024-01-20T08:59:07.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.399+0000 7f4612d9d640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4614159770 0x7f4614144c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097a0 tx=0x7f460c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.403+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c03e070 con 0x7f4614159770 2024-01-20T08:59:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.403+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c039660 con 0x7f4614159770 2024-01-20T08:59:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.403+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461415ae70 con 0x7f4614159770 2024-01-20T08:59:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.403+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c039a30 con 0x7f4614159770 2024-01-20T08:59:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.403+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461415b430 con 0x7f4614159770 2024-01-20T08:59:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.404+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f461406b540 con 0x7f4614159770 2024-01-20T08:59:07.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.424+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f460c039b90 con 0x7f4614159770 2024-01-20T08:59:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.424+0000 7f4602ffd640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f45f406af20 0x7f45f406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.424+0000 7f461259c640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f45f406af20 0x7f45f406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.425+0000 7f461259c640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f45f406af20 0x7f45f406d3e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4604011340 tx=0x7f460400b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.428+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f460c039e40 con 0x7f4614159770 2024-01-20T08:59:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.428+0000 7f4602ffd640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7f45f4074520 0x7f45f40769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.428+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 --> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45f4077070 con 0x7f45f4074520 2024-01-20T08:59:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.428+0000 7f4611d9b640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7f45f4074520 0x7f45f40769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.428+0000 7f4611d9b640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7f45f4074520 0x7f45f40769a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.429+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f460c0770e0 con 0x7f4614159770 2024-01-20T08:59:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.429+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== osd.5 v2:172.21.15.116:6816/4162650663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f45f4077070 con 0x7f45f4074520 2024-01-20T08:59:07.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.443+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 --> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f461415b5c0 con 0x7f45f4074520 2024-01-20T08:59:07.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.443+0000 7f4602ffd640 1 -- 172.21.15.87:0/3968162820 <== osd.5 v2:172.21.15.116:6816/4162650663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f461415b5c0 con 0x7f45f4074520 2024-01-20T08:59:07.444 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:07.444 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:07.444 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:07.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.443+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7f45f4074520 msgr2=0x7f45f40769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.443+0000 7f4618bb7640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:6816/4162650663,v1:172.21.15.116:6817/4162650663] conn(0x7f45f4074520 0x7f45f40769a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f45f406af20 msgr2=0x7f45f406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f45f406af20 0x7f45f406d3e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4604011340 tx=0x7f460400b040 comp rx=0 tx=0).stop 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4614159770 msgr2=0x7f4614144c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 --2- 172.21.15.87:0/3968162820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4614159770 0x7f4614144c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097a0 tx=0x7f460c0046c0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4612d9d640 1 -- 172.21.15.87:0/3968162820 reap_dead start 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 shutdown_connections 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 >> 172.21.15.87:0/3968162820 conn(0x7f4614071ff0 msgr2=0x7f46140c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 shutdown_connections 2024-01-20T08:59:07.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.444+0000 7f4618bb7640 1 -- 172.21.15.87:0/3968162820 wait complete. 2024-01-20T08:59:07.455 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-20T08:59:07.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.559+0000 7f48801d7640 1 Processor -- start 2024-01-20T08:59:07.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.559+0000 7f48801d7640 1 -- start start 2024-01-20T08:59:07.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.560+0000 7f48801d7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48781513a0 0x7f48780663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.560+0000 7f48801d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4878151780 0x7f487806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.560+0000 7f48801d7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48780668f0 con 0x7f487806bbc0 2024-01-20T08:59:07.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.560+0000 7f48801d7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4878158470 con 0x7f48781513a0 2024-01-20T08:59:07.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.560+0000 7f48801d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4878157530 con 0x7f4878151780 2024-01-20T08:59:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.561+0000 7f487d772640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4878151780 0x7f487806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.561+0000 7f487e774640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f487806bbc0 0x7f4878071990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34796/0 (socket says 172.21.15.87:34796) 2024-01-20T08:59:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.561+0000 7f487df73640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48781513a0 0x7f48780663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.561+0000 7f487e774640 1 -- 172.21.15.87:0/1217847120 learned_addr learned my addr 172.21.15.87:0/1217847120 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.563+0000 7f487d772640 1 -- 172.21.15.87:0/1217847120 >> v2:172.21.15.116:3300/0 conn(0x7f48781513a0 msgr2=0x7f48780663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.563+0000 7f487d772640 1 --2- 172.21.15.87:0/1217847120 >> v2:172.21.15.116:3300/0 conn(0x7f48781513a0 0x7f48780663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.563+0000 7f487d772640 1 -- 172.21.15.87:0/1217847120 >> v1:172.21.15.87:6789/0 conn(0x7f487806bbc0 legacy=0x7f4878071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.563+0000 7f487d772640 1 -- 172.21.15.87:0/1217847120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48781577f0 con 0x7f4878151780 2024-01-20T08:59:07.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.566+0000 7f487d772640 1 --2- 172.21.15.87:0/1217847120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4878151780 0x7f487806a5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48680097a0 tx=0x7f486802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4d702aeecb2b5c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f487cf71640 1 -- 172.21.15.87:0/1217847120 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486802f050 con 0x7f4878151780 2024-01-20T08:59:07.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f487cf71640 1 -- 172.21.15.87:0/1217847120 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4868031700 con 0x7f4878151780 2024-01-20T08:59:07.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f487cf71640 1 -- 172.21.15.87:0/1217847120 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868031a90 con 0x7f4878151780 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f48801d7640 1 -- 172.21.15.87:0/1217847120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4878151780 msgr2=0x7f487806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f48801d7640 1 --2- 172.21.15.87:0/1217847120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4878151780 0x7f487806a5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48680097a0 tx=0x7f486802b7a0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f48801d7640 1 -- 172.21.15.87:0/1217847120 shutdown_connections 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f48801d7640 1 --2- 172.21.15.87:0/1217847120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4878151780 0x7f487806a5b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f48801d7640 1 --2- 172.21.15.87:0/1217847120 >> v2:172.21.15.116:3300/0 conn(0x7f48781513a0 0x7f48780663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.570+0000 7f48801d7640 1 -- 172.21.15.87:0/1217847120 >> 172.21.15.87:0/1217847120 conn(0x7f48780cb310 msgr2=0x7f48780cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.571+0000 7f48801d7640 1 -- 172.21.15.87:0/1217847120 shutdown_connections 2024-01-20T08:59:07.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.571+0000 7f48801d7640 1 -- 172.21.15.87:0/1217847120 wait complete. 2024-01-20T08:59:07.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 Processor -- start 2024-01-20T08:59:07.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 -- start start 2024-01-20T08:59:07.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487806bbc0 0x7f48781de020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48781513a0 0x7f48781e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 0x7f48780c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48781ee120 con 0x7f4878151780 2024-01-20T08:59:07.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48781e0800 con 0x7f487806bbc0 2024-01-20T08:59:07.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.572+0000 7f48801d7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48781e4150 con 0x7f48781513a0 2024-01-20T08:59:07.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.573+0000 7f487e774640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 0x7f48780c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.573+0000 7f487d772640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48781513a0 0x7f48781e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.573+0000 7f487df73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487806bbc0 0x7f48781de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.573+0000 7f487e774640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 0x7f48780c0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51532/0 (socket says 172.21.15.87:51532) 2024-01-20T08:59:07.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.573+0000 7f487e774640 1 -- 172.21.15.87:0/2486385480 learned_addr learned my addr 172.21.15.87:0/2486385480 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:07.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.577+0000 7f487e774640 1 -- 172.21.15.87:0/2486385480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48781513a0 msgr2=0x7f48781e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.577+0000 7f487e774640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48781513a0 0x7f48781e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.577+0000 7f487e774640 1 -- 172.21.15.87:0/2486385480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487806bbc0 msgr2=0x7f48781de020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:07.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.577+0000 7f487e774640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487806bbc0 0x7f48781de020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:07.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.577+0000 7f487e774640 1 -- 172.21.15.87:0/2486385480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4868009400 con 0x7f4878151780 2024-01-20T08:59:07.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.579+0000 7f487e774640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 0x7f48780c0fb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f487000c3c0 tx=0x7f487000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.581+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870005ce0 con 0x7f4878151780 2024-01-20T08:59:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.581+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48781e4410 con 0x7f4878151780 2024-01-20T08:59:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.581+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48781ed640 con 0x7f4878151780 2024-01-20T08:59:07.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.582+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4878068110 con 0x7f4878151780 2024-01-20T08:59:07.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.584+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870005e40 con 0x7f4878151780 2024-01-20T08:59:07.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.584+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700072d0 con 0x7f4878151780 2024-01-20T08:59:07.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.834+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f48700074f0 con 0x7f4878151780 2024-01-20T08:59:07.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.834+0000 7f486e7fc640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f485c06aed0 0x7f485c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:07.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.834+0000 7f487df73640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f485c06aed0 0x7f485c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.835+0000 7f487df73640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f485c06aed0 0x7f485c06d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48740113e0 tx=0x7f487400b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.850+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4870057f60 con 0x7f4878151780 2024-01-20T08:59:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:07.850+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f48700542a0 con 0x7f4878151780 2024-01-20T08:59:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.025+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f487814e7d0 con 0x7f485c06aed0 2024-01-20T08:59:08.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.025+0000 7f486e7fc640 1 -- 172.21.15.87:0/2486385480 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f487814e7d0 con 0x7f485c06aed0 2024-01-20T08:59:08.027 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 5 to deep-scrub 2024-01-20T08:59:08.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f485c06aed0 msgr2=0x7f485c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f485c06aed0 0x7f485c06d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48740113e0 tx=0x7f487400b040 comp rx=0 tx=0).stop 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 msgr2=0x7f48780c0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 0x7f48780c0fb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f487000c3c0 tx=0x7f487000c880 comp rx=0 tx=0).stop 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 shutdown_connections 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f485c06aed0 0x7f485c06d390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4878151780 0x7f48780c0fb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48781513a0 0x7f48781e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 --2- 172.21.15.87:0/2486385480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487806bbc0 0x7f48781de020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 >> 172.21.15.87:0/2486385480 conn(0x7f48780cb310 msgr2=0x7f487806f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 shutdown_connections 2024-01-20T08:59:08.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.027+0000 7f48801d7640 1 -- 172.21.15.87:0/2486385480 wait complete. 2024-01-20T08:59:08.039 INFO:tasks.ceph:Scrubbing osd.6 2024-01-20T08:59:08.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:08.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.145+0000 7f20779f8640 1 Processor -- start 2024-01-20T08:59:08.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.145+0000 7f20779f8640 1 -- start start 2024-01-20T08:59:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.146+0000 7f20779f8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2070150350 0x7f20700c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.146+0000 7f20779f8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 0x7f2070156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.146+0000 7f20779f8640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2070071910 con 0x7f20701598a0 2024-01-20T08:59:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.146+0000 7f20779f8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2070159d30 con 0x7f2070150350 2024-01-20T08:59:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.146+0000 7f20779f8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2070150730 con 0x7f20700bff90 2024-01-20T08:59:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.147+0000 7f2074f93640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 0x7f2070156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.147+0000 7f2074f93640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 0x7f2070156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:52924/0 (socket says 172.21.15.87:52924) 2024-01-20T08:59:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.147+0000 7f2074f93640 1 -- 172.21.15.87:0/2251672130 learned_addr learned my addr 172.21.15.87:0/2251672130 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.147+0000 7f2075794640 1 --2- 172.21.15.87:0/2251672130 >> v2:172.21.15.116:3300/0 conn(0x7f2070150350 0x7f20700c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.149+0000 7f2074f93640 1 -- 172.21.15.87:0/2251672130 >> v2:172.21.15.116:3300/0 conn(0x7f2070150350 msgr2=0x7f20700c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:08.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.149+0000 7f2074f93640 1 --2- 172.21.15.87:0/2251672130 >> v2:172.21.15.116:3300/0 conn(0x7f2070150350 0x7f20700c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.149+0000 7f2074f93640 1 -- 172.21.15.87:0/2251672130 >> v1:172.21.15.116:6789/0 conn(0x7f20701598a0 legacy=0x7f20700cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.149+0000 7f2074f93640 1 -- 172.21.15.87:0/2251672130 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070071b60 con 0x7f20700bff90 2024-01-20T08:59:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.152+0000 7f2074f93640 1 --2- 172.21.15.87:0/2251672130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 0x7f2070156ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f206000bb10 tx=0x7f206002e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db2b7f449cd135b server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.155+0000 7f2067fff640 1 -- 172.21.15.87:0/2251672130 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060036440 con 0x7f20700bff90 2024-01-20T08:59:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.155+0000 7f2067fff640 1 -- 172.21.15.87:0/2251672130 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20600365a0 con 0x7f20700bff90 2024-01-20T08:59:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.155+0000 7f2067fff640 1 -- 172.21.15.87:0/2251672130 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060036930 con 0x7f20700bff90 2024-01-20T08:59:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 -- 172.21.15.87:0/2251672130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 msgr2=0x7f2070156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 --2- 172.21.15.87:0/2251672130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 0x7f2070156ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f206000bb10 tx=0x7f206002e150 comp rx=0 tx=0).stop 2024-01-20T08:59:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 -- 172.21.15.87:0/2251672130 shutdown_connections 2024-01-20T08:59:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 --2- 172.21.15.87:0/2251672130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20700bff90 0x7f2070156ea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 --2- 172.21.15.87:0/2251672130 >> v2:172.21.15.116:3300/0 conn(0x7f2070150350 0x7f20700c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 -- 172.21.15.87:0/2251672130 >> 172.21.15.87:0/2251672130 conn(0x7f207006d510 msgr2=0x7f20700663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 -- 172.21.15.87:0/2251672130 shutdown_connections 2024-01-20T08:59:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.156+0000 7f20779f8640 1 -- 172.21.15.87:0/2251672130 wait complete. 2024-01-20T08:59:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.157+0000 7f20779f8640 1 Processor -- start 2024-01-20T08:59:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.157+0000 7f20779f8640 1 -- start start 2024-01-20T08:59:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.158+0000 7f20779f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20700bff90 0x7f2070155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.158+0000 7f20779f8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2070150350 0x7f20700c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.158+0000 7f20779f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20701598a0 0x7f20701ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.158+0000 7f20779f8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20701efaf0 con 0x7f20700bff90 2024-01-20T08:59:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.158+0000 7f20779f8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2070155050 con 0x7f20701598a0 2024-01-20T08:59:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.158+0000 7f20779f8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2070157470 con 0x7f2070150350 2024-01-20T08:59:08.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.159+0000 7f2075f95640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20701598a0 0x7f20701ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.159+0000 7f2074f93640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2070150350 0x7f20700c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.159+0000 7f2075f95640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20701598a0 0x7f20701ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41486/0 (socket says 172.21.15.87:41486) 2024-01-20T08:59:08.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.159+0000 7f2074f93640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2070150350 0x7f20700c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:52936/0 (socket says 172.21.15.87:52936) 2024-01-20T08:59:08.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.159+0000 7f2075f95640 1 -- 172.21.15.87:0/1476936676 learned_addr learned my addr 172.21.15.87:0/1476936676 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:08.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.160+0000 7f2075794640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20700bff90 0x7f2070155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.162+0000 7f2075f95640 1 -- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2070150350 msgr2=0x7f20700c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.162+0000 7f2075f95640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2070150350 0x7f20700c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.162+0000 7f2075f95640 1 -- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20700bff90 msgr2=0x7f2070155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.162+0000 7f2075f95640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20700bff90 0x7f2070155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.162+0000 7f2075f95640 1 -- 172.21.15.87:0/1476936676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206000b770 con 0x7f20701598a0 2024-01-20T08:59:08.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.164+0000 7f2075f95640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20701598a0 0x7f20701ef5b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f206800c390 tx=0x7f206800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.168+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20680052d0 con 0x7f20701598a0 2024-01-20T08:59:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.168+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068005430 con 0x7f20701598a0 2024-01-20T08:59:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.168+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068003810 con 0x7f20701598a0 2024-01-20T08:59:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.168+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2070157730 con 0x7f20701598a0 2024-01-20T08:59:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.168+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20701555d0 con 0x7f20701598a0 2024-01-20T08:59:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.169+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2070156980 con 0x7f20701598a0 2024-01-20T08:59:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.187+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2068003970 con 0x7f20701598a0 2024-01-20T08:59:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.187+0000 7f2065ffb640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f205406af20 0x7f205406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.187+0000 7f2075794640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f205406af20 0x7f205406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.188+0000 7f2075794640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f205406af20 0x7f205406d3e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f206c0113e0 tx=0x7f206c00b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.190+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f2068057e20 con 0x7f20701598a0 2024-01-20T08:59:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.190+0000 7f2065ffb640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f2054074520 0x7f20540769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.190+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 --> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2054077070 con 0x7f2054074520 2024-01-20T08:59:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.190+0000 7f2074f93640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f2054074520 0x7f20540769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.191+0000 7f2074f93640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f2054074520 0x7f20540769a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.191+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== osd.6 v2:172.21.15.116:6821/3458193509 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2054077070 con 0x7f2054074520 2024-01-20T08:59:08.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.192+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2068054160 con 0x7f20701598a0 2024-01-20T08:59:08.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.204+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 --> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f20701566a0 con 0x7f2054074520 2024-01-20T08:59:08.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f2065ffb640 1 -- 172.21.15.87:0/1476936676 <== osd.6 v2:172.21.15.116:6821/3458193509 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2060036890 con 0x7f2054074520 2024-01-20T08:59:08.206 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:08.206 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:08.206 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f2054074520 msgr2=0x7f20540769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:6821/3458193509,v1:172.21.15.116:6823/3458193509] conn(0x7f2054074520 0x7f20540769a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f205406af20 msgr2=0x7f205406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f205406af20 0x7f205406d3e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f206c0113e0 tx=0x7f206c00b040 comp rx=0 tx=0).stop 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20701598a0 msgr2=0x7f20701ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 --2- 172.21.15.87:0/1476936676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20701598a0 0x7f20701ef5b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f206800c390 tx=0x7f206800c850 comp rx=0 tx=0).stop 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f2075f95640 1 -- 172.21.15.87:0/1476936676 reap_dead start 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 shutdown_connections 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 >> 172.21.15.87:0/1476936676 conn(0x7f207006d510 msgr2=0x7f207014c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 shutdown_connections 2024-01-20T08:59:08.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.205+0000 7f20779f8640 1 -- 172.21.15.87:0/1476936676 wait complete. 2024-01-20T08:59:08.216 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-20T08:59:08.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.319+0000 7fbfe39e5640 1 Processor -- start 2024-01-20T08:59:08.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.319+0000 7fbfe39e5640 1 -- start start 2024-01-20T08:59:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe39e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe39e5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbfdc0c81f0 0x7fbfdc160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe39e5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfdc156720 con 0x7fbfdc14cc40 2024-01-20T08:59:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe39e5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc1572c0 con 0x7fbfdc0c81f0 2024-01-20T08:59:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe39e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc072b30 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe0f80640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe0f80640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:52938/0 (socket says 172.21.15.87:52938) 2024-01-20T08:59:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.320+0000 7fbfe0f80640 1 -- 172.21.15.87:0/3248949768 learned_addr learned my addr 172.21.15.87:0/3248949768 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.321+0000 7fbfe1f82640 1 --2- 172.21.15.87:0/3248949768 >> v2:172.21.15.116:3300/0 conn(0x7fbfdc0c81f0 0x7fbfdc160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.323+0000 7fbfe0f80640 1 -- 172.21.15.87:0/3248949768 >> v2:172.21.15.116:3300/0 conn(0x7fbfdc0c81f0 msgr2=0x7fbfdc160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.323+0000 7fbfe0f80640 1 --2- 172.21.15.87:0/3248949768 >> v2:172.21.15.116:3300/0 conn(0x7fbfdc0c81f0 0x7fbfdc160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.323+0000 7fbfe0f80640 1 -- 172.21.15.87:0/3248949768 >> v1:172.21.15.116:6789/0 conn(0x7fbfdc14cc40 legacy=0x7fbfdc156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.323+0000 7fbfe0f80640 1 -- 172.21.15.87:0/3248949768 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0c19c0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.325+0000 7fbfe0f80640 1 --2- 172.21.15.87:0/3248949768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c7cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009870 tx=0x7fbfcc02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a190000e63a9e0f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.328+0000 7fbfd3fff640 1 -- 172.21.15.87:0/3248949768 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc02f840 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.328+0000 7fbfd3fff640 1 -- 172.21.15.87:0/3248949768 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc02f9a0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.328+0000 7fbfd3fff640 1 -- 172.21.15.87:0/3248949768 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc02fd30 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 -- 172.21.15.87:0/3248949768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 msgr2=0x7fbfdc0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/3248949768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c7cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009870 tx=0x7fbfcc02ba20 comp rx=0 tx=0).stop 2024-01-20T08:59:08.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 -- 172.21.15.87:0/3248949768 shutdown_connections 2024-01-20T08:59:08.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/3248949768 >> v2:172.21.15.116:3300/0 conn(0x7fbfdc0c81f0 0x7fbfdc160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/3248949768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c7cb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 -- 172.21.15.87:0/3248949768 >> 172.21.15.87:0/3248949768 conn(0x7fbfdc06a9a0 msgr2=0x7fbfdc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 -- 172.21.15.87:0/3248949768 shutdown_connections 2024-01-20T08:59:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.329+0000 7fbfe39e5640 1 -- 172.21.15.87:0/3248949768 wait complete. 2024-01-20T08:59:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.330+0000 7fbfe39e5640 1 Processor -- start 2024-01-20T08:59:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.330+0000 7fbfe39e5640 1 -- start start 2024-01-20T08:59:08.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe39e5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe39e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0c81f0 0x7fbfdc14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe39e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfdc14cc40 0x7fbfdc1ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe39e5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0c4b50 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe39e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0c3a50 con 0x7fbfdc14cc40 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe39e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1f0bc0 con 0x7fbfdc0c81f0 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.331+0000 7fbfe1781640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.332+0000 7fbfe1f82640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfdc14cc40 0x7fbfdc1ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.332+0000 7fbfe0f80640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0c81f0 0x7fbfdc14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.332+0000 7fbfe1f82640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfdc14cc40 0x7fbfdc1ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:41508/0 (socket says 172.21.15.87:41508) 2024-01-20T08:59:08.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.332+0000 7fbfe1f82640 1 -- 172.21.15.87:0/2481274130 learned_addr learned my addr 172.21.15.87:0/2481274130 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:08.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.335+0000 7fbfe1781640 1 -- 172.21.15.87:0/2481274130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0c81f0 msgr2=0x7fbfdc14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.335+0000 7fbfe1781640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0c81f0 0x7fbfdc14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.335+0000 7fbfe1781640 1 -- 172.21.15.87:0/2481274130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfdc14cc40 msgr2=0x7fbfdc1ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.335+0000 7fbfe1781640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfdc14cc40 0x7fbfdc1ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.335+0000 7fbfe1781640 1 -- 172.21.15.87:0/2481274130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc0094d0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.338+0000 7fbfe1781640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c4610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c340 tx=0x7fbfd800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.341+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80052d0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.341+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8005450 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.341+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc1f10b0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.341+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8007260 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.342+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc1534a0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.343+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc0c5b10 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.360+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbfd80073c0 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.360+0000 7fbfd1ffb640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbfc006af50 0x7fbfc006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.360+0000 7fbfe0f80640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbfc006af50 0x7fbfc006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.361+0000 7fbfe0f80640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbfc006af50 0x7fbfc006d410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009870 tx=0x7fbfcc008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.364+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fbfd8057a00 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.599+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbfd8053d40 con 0x7fbfdc0bfe70 2024-01-20T08:59:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.775+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfdc0c1f40 con 0x7fbfc006af50 2024-01-20T08:59:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.775+0000 7fbfd1ffb640 1 -- 172.21.15.87:0/2481274130 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbfcc02fb90 con 0x7fbfc006af50 2024-01-20T08:59:08.776 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 6 to deep-scrub 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbfc006af50 msgr2=0x7fbfc006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbfc006af50 0x7fbfc006d410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009870 tx=0x7fbfcc008030 comp rx=0 tx=0).stop 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfdc0bfe70 msgr2=0x7fbfdc0c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c4610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c340 tx=0x7fbfd800c800 comp rx=0 tx=0).stop 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 shutdown_connections 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fbfc006af50 0x7fbfc006d410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfdc14cc40 0x7fbfdc1ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfdc0c81f0 0x7fbfdc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 --2- 172.21.15.87:0/2481274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfdc0bfe70 0x7fbfdc0c4610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 >> 172.21.15.87:0/2481274130 conn(0x7fbfdc06a9a0 msgr2=0x7fbfdc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 shutdown_connections 2024-01-20T08:59:08.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.777+0000 7fbfe39e5640 1 -- 172.21.15.87:0/2481274130 wait complete. 2024-01-20T08:59:08.790 INFO:tasks.ceph:Scrubbing osd.7 2024-01-20T08:59:08.791 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-20T08:59:08.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.894+0000 7f0483578640 1 Processor -- start 2024-01-20T08:59:08.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.894+0000 7f0483578640 1 -- start start 2024-01-20T08:59:08.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.895+0000 7f0483578640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c1538e0 0x7f047c071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.895+0000 7f0483578640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f047c0c8ff0 0x7f047c159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.895+0000 7f0483578640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0c7ff0 con 0x7f047c159b70 2024-01-20T08:59:08.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.895+0000 7f0483578640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047c15daf0 con 0x7f047c0c8ff0 2024-01-20T08:59:08.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.895+0000 7f0483578640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0c7d10 con 0x7f047c1538e0 2024-01-20T08:59:08.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.896+0000 7f0481b15640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f047c159b70 0x7f047c0c9680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34804/0 (socket says 172.21.15.87:34804) 2024-01-20T08:59:08.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.896+0000 7f0481b15640 1 -- 172.21.15.87:0/992056908 learned_addr learned my addr 172.21.15.87:0/992056908 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:08.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.896+0000 7f0480b13640 1 --2- 172.21.15.87:0/992056908 >> v2:172.21.15.116:3300/0 conn(0x7f047c0c8ff0 0x7f047c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.896+0000 7f0481314640 1 --2- 172.21.15.87:0/992056908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c1538e0 0x7f047c071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.898+0000 7f0473fff640 1 -- 172.21.15.87:0/992056908 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530822805 0 0) 0x7f047c0c7ff0 con 0x7f047c159b70 2024-01-20T08:59:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.898+0000 7f0473fff640 1 -- 172.21.15.87:0/992056908 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0468003520 con 0x7f047c159b70 2024-01-20T08:59:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.898+0000 7f0481314640 1 -- 172.21.15.87:0/992056908 >> v2:172.21.15.116:3300/0 conn(0x7f047c0c8ff0 msgr2=0x7f047c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.898+0000 7f0481314640 1 --2- 172.21.15.87:0/992056908 >> v2:172.21.15.116:3300/0 conn(0x7f047c0c8ff0 0x7f047c159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.898+0000 7f0481314640 1 -- 172.21.15.87:0/992056908 >> v1:172.21.15.87:6789/0 conn(0x7f047c159b70 legacy=0x7f047c0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.898+0000 7f0481314640 1 -- 172.21.15.87:0/992056908 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c15cd30 con 0x7f047c1538e0 2024-01-20T08:59:08.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.900+0000 7f0481314640 1 --2- 172.21.15.87:0/992056908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c1538e0 0x7f047c071450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f047802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fbd1c7a3272419 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.902+0000 7f0473fff640 1 -- 172.21.15.87:0/992056908 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047802cce0 con 0x7f047c1538e0 2024-01-20T08:59:08.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.904+0000 7f0473fff640 1 -- 172.21.15.87:0/992056908 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047802ce40 con 0x7f047c1538e0 2024-01-20T08:59:08.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.904+0000 7f0473fff640 1 -- 172.21.15.87:0/992056908 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478034640 con 0x7f047c1538e0 2024-01-20T08:59:08.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 -- 172.21.15.87:0/992056908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c1538e0 msgr2=0x7f047c071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 --2- 172.21.15.87:0/992056908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c1538e0 0x7f047c071450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f047802bf30 comp rx=0 tx=0).stop 2024-01-20T08:59:08.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 -- 172.21.15.87:0/992056908 shutdown_connections 2024-01-20T08:59:08.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 --2- 172.21.15.87:0/992056908 >> v2:172.21.15.116:3300/0 conn(0x7f047c0c8ff0 0x7f047c159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 --2- 172.21.15.87:0/992056908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c1538e0 0x7f047c071450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 -- 172.21.15.87:0/992056908 >> 172.21.15.87:0/992056908 conn(0x7f047c06a5b0 msgr2=0x7f047c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 -- 172.21.15.87:0/992056908 shutdown_connections 2024-01-20T08:59:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.905+0000 7f0483578640 1 -- 172.21.15.87:0/992056908 wait complete. 2024-01-20T08:59:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.906+0000 7f0483578640 1 Processor -- start 2024-01-20T08:59:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.906+0000 7f0483578640 1 -- start start 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.907+0000 7f0483578640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f047c0c8ff0 0x7f047c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.907+0000 7f0483578640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f047c1538e0 0x7f047c1e3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.907+0000 7f0483578640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c159b70 0x7f047c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.907+0000 7f0483578640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0cc140 con 0x7f047c1538e0 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.907+0000 7f0483578640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1e3550 con 0x7f047c0c8ff0 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.907+0000 7f0483578640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0cc560 con 0x7f047c159b70 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.908+0000 7f0480b13640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f047c1538e0 0x7f047c1e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.908+0000 7f0481b15640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c159b70 0x7f047c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.908+0000 7f0481314640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f047c0c8ff0 0x7f047c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.908+0000 7f0480b13640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f047c1538e0 0x7f047c1e3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51564/0 (socket says 172.21.15.87:51564) 2024-01-20T08:59:08.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.908+0000 7f0480b13640 1 -- 172.21.15.87:0/2204765472 learned_addr learned my addr 172.21.15.87:0/2204765472 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:08.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.911+0000 7f0480b13640 1 -- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c159b70 msgr2=0x7f047c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.911+0000 7f0480b13640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f047c159b70 0x7f047c0cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.911+0000 7f0480b13640 1 -- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f047c0c8ff0 msgr2=0x7f047c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.911+0000 7f0480b13640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f047c0c8ff0 0x7f047c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.911+0000 7f0480b13640 1 -- 172.21.15.87:0/2204765472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478009a70 con 0x7f047c1538e0 2024-01-20T08:59:08.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.913+0000 7f0480b13640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f047c1538e0 0x7f047c1e3010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e870 tx=0x7f046c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.917+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c0094f0 con 0x7f047c1538e0 2024-01-20T08:59:08.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.917+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00c940 con 0x7f047c1538e0 2024-01-20T08:59:08.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.917+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00ccf0 con 0x7f047c1538e0 2024-01-20T08:59:08.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.917+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c0cb200 con 0x7f047c1538e0 2024-01-20T08:59:08.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.917+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c0cb7a0 con 0x7f047c1538e0 2024-01-20T08:59:08.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.918+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f047c15d5d0 con 0x7f047c1538e0 2024-01-20T08:59:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.937+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f046c013470 con 0x7f047c1538e0 2024-01-20T08:59:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.937+0000 7f0471ffb640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f046006aed0 0x7f046006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.937+0000 7f0481314640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f046006aed0 0x7f046006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.938+0000 7f0481314640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f046006aed0 0x7f046006d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0478036000 tx=0x7f0478030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.940+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f046c058610 con 0x7f047c1538e0 2024-01-20T08:59:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.941+0000 7f0471ffb640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f0460074450 0x7f04600768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.941+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 --> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0460076fa0 con 0x7f0460074450 2024-01-20T08:59:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.941+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f046c054950 con 0x7f047c1538e0 2024-01-20T08:59:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.941+0000 7f0481b15640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f0460074450 0x7f04600768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.941+0000 7f0481b15640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f0460074450 0x7f04600768d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:08.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.942+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== osd.7 v2:172.21.15.116:6800/1436327026 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0460076fa0 con 0x7f0460074450 2024-01-20T08:59:08.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.957+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 --> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f047c15d2f0 con 0x7f0460074450 2024-01-20T08:59:08.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.957+0000 7f0471ffb640 1 -- 172.21.15.87:0/2204765472 <== osd.7 v2:172.21.15.116:6800/1436327026 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f04740047e0 con 0x7f0460074450 2024-01-20T08:59:08.959 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-01-20T08:59:08.959 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-20T08:59:08.959 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f0460074450 msgr2=0x7f04600768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.116:6800/1436327026,v1:172.21.15.116:6801/1436327026] conn(0x7f0460074450 0x7f04600768d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f046006aed0 msgr2=0x7f046006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f046006aed0 0x7f046006d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0478036000 tx=0x7f0478030000 comp rx=0 tx=0).stop 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f047c1538e0 msgr2=0x7f047c1e3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 --2- 172.21.15.87:0/2204765472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f047c1538e0 0x7f047c1e3010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e870 tx=0x7f046c00ed30 comp rx=0 tx=0).stop 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0481b15640 1 -- 172.21.15.87:0/2204765472 reap_dead start 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 shutdown_connections 2024-01-20T08:59:08.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 >> 172.21.15.87:0/2204765472 conn(0x7f047c06a5b0 msgr2=0x7f047c15c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:08.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 shutdown_connections 2024-01-20T08:59:08.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:08.958+0000 7f0483578640 1 -- 172.21.15.87:0/2204765472 wait complete. 2024-01-20T08:59:08.970 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-20T08:59:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.075+0000 7fe981c07640 1 Processor -- start 2024-01-20T08:59:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.075+0000 7fe981c07640 1 -- start start 2024-01-20T08:59:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.076+0000 7fe981c07640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe97c15a1b0 0x7fe97c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.076+0000 7fe981c07640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c153a40 con 0x7fe97c0c2d80 2024-01-20T08:59:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.076+0000 7fe981c07640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c0c3a60 con 0x7fe97c153660 2024-01-20T08:59:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.076+0000 7fe981c07640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c06d420 con 0x7fe97c15a1b0 2024-01-20T08:59:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.076+0000 7fe97b7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe97c153660 0x7fe97c0c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35970/0 (socket says 172.21.15.87:35970) 2024-01-20T08:59:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.076+0000 7fe97b7fe640 1 -- 172.21.15.87:0/3109403387 learned_addr learned my addr 172.21.15.87:0/3109403387 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:09.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.085+0000 7fe97affd640 1 --2- 172.21.15.87:0/3109403387 >> v2:172.21.15.87:3300/0 conn(0x7fe97c15a1b0 0x7fe97c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.088+0000 7fe97affd640 1 -- 172.21.15.87:0/3109403387 >> v1:172.21.15.116:6789/0 conn(0x7fe97c153660 legacy=0x7fe97c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.088+0000 7fe97affd640 1 -- 172.21.15.87:0/3109403387 >> v1:172.21.15.87:6789/0 conn(0x7fe97c0c2d80 legacy=0x7fe97c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.088+0000 7fe97affd640 1 -- 172.21.15.87:0/3109403387 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c0c95d0 con 0x7fe97c15a1b0 2024-01-20T08:59:09.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.093+0000 7fe97affd640 1 --2- 172.21.15.87:0/3109403387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c0c97c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe97402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f41ac82a461ccc server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:09.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe97a7fc640 1 -- 172.21.15.87:0/3109403387 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97402cd00 con 0x7fe97c15a1b0 2024-01-20T08:59:09.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe97a7fc640 1 -- 172.21.15.87:0/3109403387 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97402ce60 con 0x7fe97c15a1b0 2024-01-20T08:59:09.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe97a7fc640 1 -- 172.21.15.87:0/3109403387 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974034660 con 0x7fe97c15a1b0 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 -- 172.21.15.87:0/3109403387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 msgr2=0x7fe97c0c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 --2- 172.21.15.87:0/3109403387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c0c97c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe97402bf50 comp rx=0 tx=0).stop 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 -- 172.21.15.87:0/3109403387 shutdown_connections 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 --2- 172.21.15.87:0/3109403387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c0c97c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 -- 172.21.15.87:0/3109403387 >> 172.21.15.87:0/3109403387 conn(0x7fe97c06a5b0 msgr2=0x7fe97c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 -- 172.21.15.87:0/3109403387 shutdown_connections 2024-01-20T08:59:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.097+0000 7fe981c07640 1 -- 172.21.15.87:0/3109403387 wait complete. 2024-01-20T08:59:09.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.099+0000 7fe981c07640 1 Processor -- start 2024-01-20T08:59:09.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.099+0000 7fe981c07640 1 -- start start 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe981c07640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe97c0c2d80 0x7fe97c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe981c07640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe97c153660 0x7fe97c0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe981c07640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe981c07640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c14e050 con 0x7fe97c15a1b0 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe981c07640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1e6850 con 0x7fe97c153660 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe981c07640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1ef750 con 0x7fe97c0c2d80 2024-01-20T08:59:09.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.100+0000 7fe97bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.101+0000 7fe97bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51574/0 (socket says 172.21.15.87:51574) 2024-01-20T08:59:09.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.101+0000 7fe97bfff640 1 -- 172.21.15.87:0/1100390747 learned_addr learned my addr 172.21.15.87:0/1100390747 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.104+0000 7fe97bfff640 1 -- 172.21.15.87:0/1100390747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe97c0c2d80 msgr2=0x7fe97c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.104+0000 7fe97bfff640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe97c0c2d80 0x7fe97c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.104+0000 7fe97bfff640 1 -- 172.21.15.87:0/1100390747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe97c153660 msgr2=0x7fe97c0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.104+0000 7fe97bfff640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe97c153660 0x7fe97c0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.104+0000 7fe97bfff640 1 -- 172.21.15.87:0/1100390747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974009a70 con 0x7fe97c15a1b0 2024-01-20T08:59:09.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.106+0000 7fe97bfff640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c151b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c3c0 tx=0x7fe96c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.110+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0052d0 con 0x7fe97c15a1b0 2024-01-20T08:59:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.110+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c005430 con 0x7fe97c15a1b0 2024-01-20T08:59:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.110+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c003810 con 0x7fe97c15a1b0 2024-01-20T08:59:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.110+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c1f0570 con 0x7fe97c15a1b0 2024-01-20T08:59:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.110+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c1efcd0 con 0x7fe97c15a1b0 2024-01-20T08:59:09.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.111+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c06c980 con 0x7fe97c15a1b0 2024-01-20T08:59:09.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.130+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe96c003970 con 0x7fe97c15a1b0 2024-01-20T08:59:09.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.130+0000 7fe963fff640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe96806af00 0x7fe96806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.130+0000 7fe97b7fe640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe96806af00 0x7fe96806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.131+0000 7fe97b7fe640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe96806af00 0x7fe96806d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9700015f0 tx=0x7fe970007a40 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:09.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.365+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fe96c058250 con 0x7fe97c15a1b0 2024-01-20T08:59:09.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.365+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe96c053e90 con 0x7fe97c15a1b0 2024-01-20T08:59:09.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.555+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe97c1596d0 con 0x7fe96806af00 2024-01-20T08:59:09.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.555+0000 7fe963fff640 1 -- 172.21.15.87:0/1100390747 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe9700058b0 con 0x7fe96806af00 2024-01-20T08:59:09.556 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 7 to deep-scrub 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.557+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe96806af00 msgr2=0x7fe96806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.557+0000 7fe981c07640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe96806af00 0x7fe96806d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9700015f0 tx=0x7fe970007a40 comp rx=0 tx=0).stop 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.557+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 msgr2=0x7fe97c151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c151b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c3c0 tx=0x7fe96c00c880 comp rx=0 tx=0).stop 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 shutdown_connections 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe96806af00 0x7fe96806d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97c15a1b0 0x7fe97c151b50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe97c153660 0x7fe97c0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 --2- 172.21.15.87:0/1100390747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe97c0c2d80 0x7fe97c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 >> 172.21.15.87:0/1100390747 conn(0x7fe97c06a5b0 msgr2=0x7fe97c0c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 shutdown_connections 2024-01-20T08:59:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.558+0000 7fe981c07640 1 -- 172.21.15.87:0/1100390747 wait complete. 2024-01-20T08:59:09.570 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:59:09.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.672+0000 7f9fe55ec640 1 Processor -- start 2024-01-20T08:59:09.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.672+0000 7f9fe55ec640 1 -- start start 2024-01-20T08:59:09.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fe55ec640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe0153ad0 0x7f9fe00cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fe55ec640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9fe015d6f0 0x7f9fe00c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fe55ec640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe0153eb0 con 0x7f9fe00ca090 2024-01-20T08:59:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fe55ec640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe015a560 con 0x7f9fe015d6f0 2024-01-20T08:59:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fe55ec640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe015dc80 con 0x7f9fe0153ad0 2024-01-20T08:59:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fdf7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9fe015d6f0 0x7f9fe00c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe0153ad0 0x7f9fe00cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fd7fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9fe00ca090 0x7f9fe00ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:35984/0 (socket says 172.21.15.87:35984) 2024-01-20T08:59:09.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.673+0000 7f9fd7fff640 1 -- 172.21.15.87:0/1785311164 learned_addr learned my addr 172.21.15.87:0/1785311164 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:09.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.676+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1785311164 >> v2:172.21.15.87:3300/0 conn(0x7f9fe015d6f0 msgr2=0x7f9fe00c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:09.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.676+0000 7f9fdeffd640 1 --2- 172.21.15.87:0/1785311164 >> v2:172.21.15.87:3300/0 conn(0x7f9fe015d6f0 0x7f9fe00c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.676+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1785311164 >> v1:172.21.15.116:6789/0 conn(0x7f9fe00ca090 legacy=0x7f9fe00ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.676+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1785311164 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00cb760 con 0x7f9fe0153ad0 2024-01-20T08:59:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.679+0000 7f9fdeffd640 1 --2- 172.21.15.87:0/1785311164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe0153ad0 0x7f9fe00cb220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00097a0 tx=0x7f9fd002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a8edd6b689bb91f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:09.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.682+0000 7f9fde7fc640 1 -- 172.21.15.87:0/1785311164 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0031830 con 0x7f9fe0153ad0 2024-01-20T08:59:09.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.682+0000 7f9fde7fc640 1 -- 172.21.15.87:0/1785311164 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0031990 con 0x7f9fe0153ad0 2024-01-20T08:59:09.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.682+0000 7f9fde7fc640 1 -- 172.21.15.87:0/1785311164 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0031d20 con 0x7f9fe0153ad0 2024-01-20T08:59:09.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1785311164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe0153ad0 msgr2=0x7f9fe00cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1785311164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe0153ad0 0x7f9fe00cb220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00097a0 tx=0x7f9fd002b8d0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1785311164 shutdown_connections 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1785311164 >> v2:172.21.15.87:3300/0 conn(0x7f9fe015d6f0 0x7f9fe00c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1785311164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe0153ad0 0x7f9fe00cb220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1785311164 >> 172.21.15.87:0/1785311164 conn(0x7f9fe008af90 msgr2=0x7f9fe008b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1785311164 shutdown_connections 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.683+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1785311164 wait complete. 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.684+0000 7f9fe55ec640 1 Processor -- start 2024-01-20T08:59:09.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 -- start start 2024-01-20T08:59:09.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 0x7f9fe014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fe0153ad0 0x7f9fe0155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe015d6f0 0x7f9fe00c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe0155650 con 0x7f9fe00ca090 2024-01-20T08:59:09.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe01546b0 con 0x7f9fe0153ad0 2024-01-20T08:59:09.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.685+0000 7f9fe55ec640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00b4bc0 con 0x7f9fe015d6f0 2024-01-20T08:59:09.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.686+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 0x7f9fe014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.686+0000 7f9fdf7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe015d6f0 0x7f9fe00c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.686+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fe0153ad0 0x7f9fe0155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.686+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 0x7f9fe014ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51604/0 (socket says 172.21.15.87:51604) 2024-01-20T08:59:09.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.686+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1629892581 learned_addr learned my addr 172.21.15.87:0/1629892581 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:09.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.689+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1629892581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe015d6f0 msgr2=0x7f9fe00c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.689+0000 7f9fdeffd640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe015d6f0 0x7f9fe00c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.689+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1629892581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fe0153ad0 msgr2=0x7f9fe0155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:09.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.689+0000 7f9fdeffd640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fe0153ad0 0x7f9fe0155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:09.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.689+0000 7f9fdeffd640 1 -- 172.21.15.87:0/1629892581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0009400 con 0x7f9fe00ca090 2024-01-20T08:59:09.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.692+0000 7f9fdeffd640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 0x7f9fe014ed40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9fd002bee0 tx=0x7f9fd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:09.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.695+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd003a6f0 con 0x7f9fe00ca090 2024-01-20T08:59:09.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.695+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd003a870 con 0x7f9fe00ca090 2024-01-20T08:59:09.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.695+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd003ac20 con 0x7f9fe00ca090 2024-01-20T08:59:09.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.695+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe0156e00 con 0x7f9fe00ca090 2024-01-20T08:59:09.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.695+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe01543d0 con 0x7f9fe00ca090 2024-01-20T08:59:09.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.696+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe0159ba0 con 0x7f9fe00ca090 2024-01-20T08:59:09.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.715+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9fd004d440 con 0x7f9fe00ca090 2024-01-20T08:59:09.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.715+0000 7f9fd77fe640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fc006af50 0x7f9fc006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:09.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.716+0000 7f9fd7fff640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fc006af50 0x7f9fc006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:09.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.716+0000 7f9fd7fff640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fc006af50 0x7f9fc006d410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b4990 tx=0x7f9fc8007e80 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:09.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.718+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f9fd0079f40 con 0x7f9fe00ca090 2024-01-20T08:59:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:09.949+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9fd0076280 con 0x7f9fe00ca090 2024-01-20T08:59:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.134+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9fe0159ea0 con 0x7f9fc006af50 2024-01-20T08:59:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.143+0000 7f9fd77fe640 1 -- 172.21.15.87:0/1629892581 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+369977 (secure 0 0 0) 0x7f9fc8001460 con 0x7f9fc006af50 2024-01-20T08:59:10.148 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:10.150 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:59:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fc006af50 msgr2=0x7f9fc006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fc006af50 0x7f9fc006d410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b4990 tx=0x7f9fc8007e80 comp rx=0 tx=0).stop 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 msgr2=0x7f9fe014ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 0x7f9fe014ed40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9fd002bee0 tx=0x7f9fd00046c0 comp rx=0 tx=0).stop 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 shutdown_connections 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f9fc006af50 0x7f9fc006d410 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fe015d6f0 0x7f9fe00c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fe0153ad0 0x7f9fe0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 --2- 172.21.15.87:0/1629892581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fe00ca090 0x7f9fe014ed40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.146+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 >> 172.21.15.87:0/1629892581 conn(0x7f9fe008af90 msgr2=0x7f9fe01512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.147+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 shutdown_connections 2024-01-20T08:59:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:10.147+0000 7f9fe55ec640 1 -- 172.21.15.87:0/1629892581 wait complete. 2024-01-20T08:59:10.158 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":870,"stamp":"2024-01-20T08:59:08.654158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39627569,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":1083,"num_read_kb":1128,"num_write":12670,"num_write_kb":354190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14985,"ondisk_log_size":14985,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461136,"kb_used_data":79888,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754513584,"statfs":{"total":773094113280,"available":772621910016,"internally_reserved":0,"allocated":81805312,"data_stored":80194922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390319664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":423,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002941"},"pg_stats":[{"pgid":"2.1f","version":"27'609","reported_seq":641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:34.122861+0000","last_change":"2024-01-20T08:31:08.593217+0000","last_active":"2024-01-20T08:58:34.122861+0000","last_peered":"2024-01-20T08:58:34.122861+0000","last_clean":"2024-01-20T08:58:34.122861+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:58:34.122861+0000","last_undegraded":"2024-01-20T08:58:34.122861+0000","last_fullsized":"2024-01-20T08:58:34.122861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:26.339625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"27'27","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808166+0000","last_change":"2024-01-20T08:31:10.260181+0000","last_active":"2024-01-20T08:58:53.808166+0000","last_peered":"2024-01-20T08:58:53.808166+0000","last_clean":"2024-01-20T08:58:53.808166+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:58:53.808166+0000","last_undegraded":"2024-01-20T08:58:53.808166+0000","last_fullsized":"2024-01-20T08:58:53.808166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:18:57.335888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"27'162","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:27.698547+0000","last_change":"2024-01-20T08:31:44.811554+0000","last_active":"2024-01-20T08:58:27.698547+0000","last_peered":"2024-01-20T08:58:27.698547+0000","last_clean":"2024-01-20T08:58:27.698547+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:58:27.698547+0000","last_undegraded":"2024-01-20T08:58:27.698547+0000","last_fullsized":"2024-01-20T08:58:27.698547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":10773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'696","reported_seq":728,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.571964+0000","last_change":"2024-01-20T08:31:08.595955+0000","last_active":"2024-01-20T08:58:47.571964+0000","last_peered":"2024-01-20T08:58:47.571964+0000","last_clean":"2024-01-20T08:58:47.571964+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:58:47.571964+0000","last_undegraded":"2024-01-20T08:58:47.571964+0000","last_fullsized":"2024-01-20T08:58:47.571964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"27'37","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843652+0000","last_change":"2024-01-20T08:31:10.277553+0000","last_active":"2024-01-20T08:58:53.843652+0000","last_peered":"2024-01-20T08:58:53.843652+0000","last_clean":"2024-01-20T08:58:53.843652+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:58:53.843652+0000","last_undegraded":"2024-01-20T08:58:53.843652+0000","last_fullsized":"2024-01-20T08:58:53.843652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:02.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"27'136","reported_seq":163,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.480676+0000","last_change":"2024-01-20T08:31:16.179717+0000","last_active":"2024-01-20T08:58:23.480676+0000","last_peered":"2024-01-20T08:58:23.480676+0000","last_clean":"2024-01-20T08:58:23.480676+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:58:23.480676+0000","last_undegraded":"2024-01-20T08:58:23.480676+0000","last_fullsized":"2024-01-20T08:58:23.480676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'1352","reported_seq":1383,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.571466+0000","last_change":"2024-01-20T08:31:08.625090+0000","last_active":"2024-01-20T08:58:47.571466+0000","last_peered":"2024-01-20T08:58:47.571466+0000","last_clean":"2024-01-20T08:58:47.571466+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:58:47.571466+0000","last_undegraded":"2024-01-20T08:58:47.571466+0000","last_fullsized":"2024-01-20T08:58:47.571466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"27'31","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.885020+0000","last_change":"2024-01-20T08:31:10.281227+0000","last_active":"2024-01-20T08:58:53.885020+0000","last_peered":"2024-01-20T08:58:53.885020+0000","last_clean":"2024-01-20T08:58:53.885020+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:58:53.885020+0000","last_undegraded":"2024-01-20T08:58:53.885020+0000","last_fullsized":"2024-01-20T08:58:53.885020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"27'128","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.790452+0000","last_change":"2024-01-20T08:31:16.183321+0000","last_active":"2024-01-20T08:58:46.790452+0000","last_peered":"2024-01-20T08:58:46.790452+0000","last_clean":"2024-01-20T08:58:46.790452+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:58:46.790452+0000","last_undegraded":"2024-01-20T08:58:46.790452+0000","last_fullsized":"2024-01-20T08:58:46.790452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.560805+0000","last_change":"2024-01-20T08:31:08.624713+0000","last_active":"2024-01-20T08:58:47.560805+0000","last_peered":"2024-01-20T08:58:47.560805+0000","last_clean":"2024-01-20T08:58:47.560805+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:58:47.560805+0000","last_undegraded":"2024-01-20T08:58:47.560805+0000","last_fullsized":"2024-01-20T08:58:47.560805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"27'47","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878405+0000","last_change":"2024-01-20T08:31:10.236876+0000","last_active":"2024-01-20T08:58:53.878405+0000","last_peered":"2024-01-20T08:58:53.878405+0000","last_clean":"2024-01-20T08:58:53.878405+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:58:53.878405+0000","last_undegraded":"2024-01-20T08:58:53.878405+0000","last_fullsized":"2024-01-20T08:58:53.878405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:53:58.037147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"27'149","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.790429+0000","last_change":"2024-01-20T08:31:16.182175+0000","last_active":"2024-01-20T08:58:46.790429+0000","last_peered":"2024-01-20T08:58:46.790429+0000","last_clean":"2024-01-20T08:58:46.790429+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:58:46.790429+0000","last_undegraded":"2024-01-20T08:58:46.790429+0000","last_fullsized":"2024-01-20T08:58:46.790429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":7763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.884346+0000","last_change":"2024-01-20T08:31:10.277228+0000","last_active":"2024-01-20T08:58:53.884346+0000","last_peered":"2024-01-20T08:58:53.884346+0000","last_clean":"2024-01-20T08:58:53.884346+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:58:53.884346+0000","last_undegraded":"2024-01-20T08:58:53.884346+0000","last_fullsized":"2024-01-20T08:58:53.884346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:19.242029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'593","reported_seq":624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.320730+0000","last_change":"2024-01-20T08:31:08.625204+0000","last_active":"2024-01-20T08:58:40.320730+0000","last_peered":"2024-01-20T08:58:40.320730+0000","last_clean":"2024-01-20T08:58:40.320730+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:58:40.320730+0000","last_undegraded":"2024-01-20T08:58:40.320730+0000","last_fullsized":"2024-01-20T08:58:40.320730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:57:28.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":4387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'109","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.837239+0000","last_change":"2024-01-20T08:31:16.174832+0000","last_active":"2024-01-20T08:58:24.837239+0000","last_peered":"2024-01-20T08:58:24.837239+0000","last_clean":"2024-01-20T08:58:24.837239+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:58:24.837239+0000","last_undegraded":"2024-01-20T08:58:24.837239+0000","last_fullsized":"2024-01-20T08:58:24.837239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:43.226082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"27'35","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.840307+0000","last_change":"2024-01-20T08:31:10.289066+0000","last_active":"2024-01-20T08:58:53.840307+0000","last_peered":"2024-01-20T08:58:53.840307+0000","last_clean":"2024-01-20T08:58:53.840307+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:58:53.840307+0000","last_undegraded":"2024-01-20T08:58:53.840307+0000","last_fullsized":"2024-01-20T08:58:53.840307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:46:23.936436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294867+0000","last_change":"2024-01-20T08:31:08.594766+0000","last_active":"2024-01-20T08:58:39.294867+0000","last_peered":"2024-01-20T08:58:39.294867+0000","last_clean":"2024-01-20T08:58:39.294867+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:58:39.294867+0000","last_undegraded":"2024-01-20T08:58:39.294867+0000","last_fullsized":"2024-01-20T08:58:39.294867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"27'149","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.903049+0000","last_change":"2024-01-20T08:31:16.177985+0000","last_active":"2024-01-20T08:58:24.903049+0000","last_peered":"2024-01-20T08:58:24.903049+0000","last_clean":"2024-01-20T08:58:24.903049+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:58:24.903049+0000","last_undegraded":"2024-01-20T08:58:24.903049+0000","last_fullsized":"2024-01-20T08:58:24.903049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.926690+0000","last_change":"2024-01-20T08:31:10.235791+0000","last_active":"2024-01-20T08:58:53.926690+0000","last_peered":"2024-01-20T08:58:53.926690+0000","last_clean":"2024-01-20T08:58:53.926690+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:58:53.926690+0000","last_undegraded":"2024-01-20T08:58:53.926690+0000","last_fullsized":"2024-01-20T08:58:53.926690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'13","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:46.531366+0000","last_change":"2024-01-20T08:31:08.592606+0000","last_active":"2024-01-20T08:58:46.531366+0000","last_peered":"2024-01-20T08:58:46.531366+0000","last_clean":"2024-01-20T08:58:46.531366+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:58:46.531366+0000","last_undegraded":"2024-01-20T08:58:46.531366+0000","last_fullsized":"2024-01-20T08:58:46.531366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"27'121","reported_seq":148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.284930+0000","last_change":"2024-01-20T08:31:16.173690+0000","last_active":"2024-01-20T08:58:23.284930+0000","last_peered":"2024-01-20T08:58:23.284930+0000","last_clean":"2024-01-20T08:58:23.284930+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:58:23.284930+0000","last_undegraded":"2024-01-20T08:58:23.284930+0000","last_fullsized":"2024-01-20T08:58:23.284930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:28.845743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'32","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.847454+0000","last_change":"2024-01-20T08:31:10.237381+0000","last_active":"2024-01-20T08:58:25.847454+0000","last_peered":"2024-01-20T08:58:25.847454+0000","last_clean":"2024-01-20T08:58:25.847454+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:58:25.847454+0000","last_undegraded":"2024-01-20T08:58:25.847454+0000","last_fullsized":"2024-01-20T08:58:25.847454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:20:03.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:42.498749+0000","last_change":"2024-01-20T08:31:08.624577+0000","last_active":"2024-01-20T08:58:42.498749+0000","last_peered":"2024-01-20T08:58:42.498749+0000","last_clean":"2024-01-20T08:58:42.498749+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:58:42.498749+0000","last_undegraded":"2024-01-20T08:58:42.498749+0000","last_fullsized":"2024-01-20T08:58:42.498749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'112","reported_seq":140,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.480718+0000","last_change":"2024-01-20T08:31:16.184054+0000","last_active":"2024-01-20T08:58:23.480718+0000","last_peered":"2024-01-20T08:58:23.480718+0000","last_clean":"2024-01-20T08:58:23.480718+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:58:23.480718+0000","last_undegraded":"2024-01-20T08:58:23.480718+0000","last_fullsized":"2024-01-20T08:58:23.480718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'41","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928171+0000","last_change":"2024-01-20T08:31:10.235845+0000","last_active":"2024-01-20T08:58:53.928171+0000","last_peered":"2024-01-20T08:58:53.928171+0000","last_clean":"2024-01-20T08:58:53.928171+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:58:53.928171+0000","last_undegraded":"2024-01-20T08:58:53.928171+0000","last_fullsized":"2024-01-20T08:58:53.928171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"27'12","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:44.519111+0000","last_change":"2024-01-20T08:31:08.584656+0000","last_active":"2024-01-20T08:58:44.519111+0000","last_peered":"2024-01-20T08:58:44.519111+0000","last_clean":"2024-01-20T08:58:44.519111+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:58:44.519111+0000","last_undegraded":"2024-01-20T08:58:44.519111+0000","last_fullsized":"2024-01-20T08:58:44.519111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:51.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"27'148","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.915090+0000","last_change":"2024-01-20T08:31:16.180987+0000","last_active":"2024-01-20T08:58:24.915090+0000","last_peered":"2024-01-20T08:58:24.915090+0000","last_clean":"2024-01-20T08:58:24.915090+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:58:24.915090+0000","last_undegraded":"2024-01-20T08:58:24.915090+0000","last_fullsized":"2024-01-20T08:58:24.915090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929778+0000","last_change":"2024-01-20T08:31:10.287431+0000","last_active":"2024-01-20T08:58:53.929778+0000","last_peered":"2024-01-20T08:58:53.929778+0000","last_clean":"2024-01-20T08:58:53.929778+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:58:53.929778+0000","last_undegraded":"2024-01-20T08:58:53.929778+0000","last_fullsized":"2024-01-20T08:58:53.929778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'315","reported_seq":346,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.274403+0000","last_change":"2024-01-20T08:31:08.592672+0000","last_active":"2024-01-20T08:58:47.274403+0000","last_peered":"2024-01-20T08:58:47.274403+0000","last_clean":"2024-01-20T08:58:47.274403+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:58:47.274403+0000","last_undegraded":"2024-01-20T08:58:47.274403+0000","last_fullsized":"2024-01-20T08:58:47.274403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:44:11.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"27'148","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.953625+0000","last_change":"2024-01-20T08:31:16.178729+0000","last_active":"2024-01-20T08:58:23.953625+0000","last_peered":"2024-01-20T08:58:23.953625+0000","last_clean":"2024-01-20T08:58:23.953625+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:58:23.953625+0000","last_undegraded":"2024-01-20T08:58:23.953625+0000","last_fullsized":"2024-01-20T08:58:23.953625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":11573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.800916+0000","last_change":"2024-01-20T08:31:10.287393+0000","last_active":"2024-01-20T08:58:53.800916+0000","last_peered":"2024-01-20T08:58:53.800916+0000","last_clean":"2024-01-20T08:58:53.800916+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:58:53.800916+0000","last_undegraded":"2024-01-20T08:58:53.800916+0000","last_fullsized":"2024-01-20T08:58:53.800916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.323865+0000","last_change":"2024-01-20T08:31:08.592341+0000","last_active":"2024-01-20T08:58:39.323865+0000","last_peered":"2024-01-20T08:58:39.323865+0000","last_clean":"2024-01-20T08:58:39.323865+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:58:39.323865+0000","last_undegraded":"2024-01-20T08:58:39.323865+0000","last_fullsized":"2024-01-20T08:58:39.323865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:52.143529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"27'126","reported_seq":154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.957683+0000","last_change":"2024-01-20T08:31:16.176483+0000","last_active":"2024-01-20T08:58:22.957683+0000","last_peered":"2024-01-20T08:58:22.957683+0000","last_clean":"2024-01-20T08:58:22.957683+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:58:22.957683+0000","last_undegraded":"2024-01-20T08:58:22.957683+0000","last_fullsized":"2024-01-20T08:58:22.957683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:11:08.928517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'33","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892221+0000","last_change":"2024-01-20T08:31:10.287355+0000","last_active":"2024-01-20T08:58:53.892221+0000","last_peered":"2024-01-20T08:58:53.892221+0000","last_clean":"2024-01-20T08:58:53.892221+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:58:53.892221+0000","last_undegraded":"2024-01-20T08:58:53.892221+0000","last_fullsized":"2024-01-20T08:58:53.892221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:43.516881+0000","last_change":"2024-01-20T08:31:08.594278+0000","last_active":"2024-01-20T08:58:43.516881+0000","last_peered":"2024-01-20T08:58:43.516881+0000","last_clean":"2024-01-20T08:58:43.516881+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:58:43.516881+0000","last_undegraded":"2024-01-20T08:58:43.516881+0000","last_fullsized":"2024-01-20T08:58:43.516881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:24:30.940711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"27'181","reported_seq":209,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.853812+0000","last_change":"2024-01-20T08:31:16.161297+0000","last_active":"2024-01-20T08:58:22.853812+0000","last_peered":"2024-01-20T08:58:22.853812+0000","last_clean":"2024-01-20T08:58:22.853812+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:58:22.853812+0000","last_undegraded":"2024-01-20T08:58:22.853812+0000","last_fullsized":"2024-01-20T08:58:22.853812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:19:03.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"27'33","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.020256+0000","last_change":"2024-01-20T08:31:10.196021+0000","last_active":"2024-01-20T08:58:54.020256+0000","last_peered":"2024-01-20T08:58:54.020256+0000","last_clean":"2024-01-20T08:58:54.020256+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:58:54.020256+0000","last_undegraded":"2024-01-20T08:58:54.020256+0000","last_fullsized":"2024-01-20T08:58:54.020256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"27'13","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:40.315491+0000","last_change":"2024-01-20T08:31:08.584923+0000","last_active":"2024-01-20T08:58:40.315491+0000","last_peered":"2024-01-20T08:58:40.315491+0000","last_clean":"2024-01-20T08:58:40.315491+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:58:40.315491+0000","last_undegraded":"2024-01-20T08:58:40.315491+0000","last_fullsized":"2024-01-20T08:58:40.315491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:02:03.458227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'113","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.935521+0000","last_change":"2024-01-20T08:31:16.157205+0000","last_active":"2024-01-20T08:58:23.935521+0000","last_peered":"2024-01-20T08:58:23.935521+0000","last_clean":"2024-01-20T08:58:23.935521+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:58:23.935521+0000","last_undegraded":"2024-01-20T08:58:23.935521+0000","last_fullsized":"2024-01-20T08:58:23.935521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:39:42.350108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808962+0000","last_change":"2024-01-20T08:31:10.263075+0000","last_active":"2024-01-20T08:58:53.808962+0000","last_peered":"2024-01-20T08:58:53.808962+0000","last_clean":"2024-01-20T08:58:53.808962+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:58:53.808962+0000","last_undegraded":"2024-01-20T08:58:53.808962+0000","last_fullsized":"2024-01-20T08:58:53.808962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"27'978","reported_seq":1977,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:49.545029+0000","last_change":"2024-01-20T08:31:08.594488+0000","last_active":"2024-01-20T08:58:49.545029+0000","last_peered":"2024-01-20T08:58:49.545029+0000","last_clean":"2024-01-20T08:58:49.545029+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:58:49.545029+0000","last_undegraded":"2024-01-20T08:58:49.545029+0000","last_fullsized":"2024-01-20T08:58:49.545029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":978,"log_dups_size":0,"ondisk_log_size":978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:21:52.348650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210232,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":976,"num_write_kb":1034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"27'135","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.812343+0000","last_change":"2024-01-20T08:31:16.178084+0000","last_active":"2024-01-20T08:58:25.812343+0000","last_peered":"2024-01-20T08:58:25.812343+0000","last_clean":"2024-01-20T08:58:25.812343+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:58:25.812343+0000","last_undegraded":"2024-01-20T08:58:25.812343+0000","last_fullsized":"2024-01-20T08:58:25.812343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118,"num_write_kb":10261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"27'28","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804624+0000","last_change":"2024-01-20T08:31:10.287878+0000","last_active":"2024-01-20T08:58:53.804624+0000","last_peered":"2024-01-20T08:58:53.804624+0000","last_clean":"2024-01-20T08:58:53.804624+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:58:53.804624+0000","last_undegraded":"2024-01-20T08:58:53.804624+0000","last_fullsized":"2024-01-20T08:58:53.804624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:28:24.392295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"27'429","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.345853+0000","last_change":"2024-01-20T08:31:08.596096+0000","last_active":"2024-01-20T08:58:41.345853+0000","last_peered":"2024-01-20T08:58:41.345853+0000","last_clean":"2024-01-20T08:58:41.345853+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:58:41.345853+0000","last_undegraded":"2024-01-20T08:58:41.345853+0000","last_fullsized":"2024-01-20T08:58:41.345853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:37:36.046647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":4309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"27'129","reported_seq":156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.864992+0000","last_change":"2024-01-20T08:31:16.174193+0000","last_active":"2024-01-20T08:58:23.864992+0000","last_peered":"2024-01-20T08:58:23.864992+0000","last_clean":"2024-01-20T08:58:23.864992+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:58:23.864992+0000","last_undegraded":"2024-01-20T08:58:23.864992+0000","last_fullsized":"2024-01-20T08:58:23.864992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:23:16.148617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'32","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881046+0000","last_change":"2024-01-20T08:31:10.191354+0000","last_active":"2024-01-20T08:58:53.881046+0000","last_peered":"2024-01-20T08:58:53.881046+0000","last_clean":"2024-01-20T08:58:53.881046+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:58:53.881046+0000","last_undegraded":"2024-01-20T08:58:53.881046+0000","last_fullsized":"2024-01-20T08:58:53.881046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:46:46.486012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"27'9","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.105404+0000","last_change":"2024-01-20T08:31:08.625012+0000","last_active":"2024-01-20T08:58:33.105404+0000","last_peered":"2024-01-20T08:58:33.105404+0000","last_clean":"2024-01-20T08:58:33.105404+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:58:33.105404+0000","last_undegraded":"2024-01-20T08:58:33.105404+0000","last_fullsized":"2024-01-20T08:58:33.105404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:09.698711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":197,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:05.824186+0000","last_change":"2024-01-20T08:59:05.824213+0000","last_active":"2024-01-20T08:59:05.824186+0000","last_peered":"2024-01-20T08:59:05.824186+0000","last_clean":"2024-01-20T08:59:05.824186+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:59:05.824186+0000","last_undegraded":"2024-01-20T08:59:05.824186+0000","last_fullsized":"2024-01-20T08:59:05.824186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-01-20T08:59:05.824134+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-01-20T08:59:05.824134+0000","last_clean_scrub_stamp":"2024-01-20T08:59:05.824134+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:29:17.877275+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'156","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.838096+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:58:24.838096+0000","last_peered":"2024-01-20T08:58:24.838096+0000","last_clean":"2024-01-20T08:58:24.838096+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:58:24.838096+0000","last_undegraded":"2024-01-20T08:58:24.838096+0000","last_fullsized":"2024-01-20T08:58:24.838096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:08:09.896756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":9711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.880987+0000","last_change":"2024-01-20T08:31:10.277872+0000","last_active":"2024-01-20T08:58:53.880987+0000","last_peered":"2024-01-20T08:58:53.880987+0000","last_clean":"2024-01-20T08:58:53.880987+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:58:53.880987+0000","last_undegraded":"2024-01-20T08:58:53.880987+0000","last_fullsized":"2024-01-20T08:58:53.880987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:41:58.120757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569355+0000","last_change":"2024-01-20T08:31:08.557184+0000","last_active":"2024-01-20T08:58:45.569355+0000","last_peered":"2024-01-20T08:58:45.569355+0000","last_clean":"2024-01-20T08:58:45.569355+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:58:45.569355+0000","last_undegraded":"2024-01-20T08:58:45.569355+0000","last_fullsized":"2024-01-20T08:58:45.569355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"27'130","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.882866+0000","last_change":"2024-01-20T08:31:16.179869+0000","last_active":"2024-01-20T08:58:23.882866+0000","last_peered":"2024-01-20T08:58:23.882866+0000","last_clean":"2024-01-20T08:58:23.882866+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:58:23.882866+0000","last_undegraded":"2024-01-20T08:58:23.882866+0000","last_fullsized":"2024-01-20T08:58:23.882866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":4819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'39","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.022904+0000","last_change":"2024-01-20T08:31:10.235840+0000","last_active":"2024-01-20T08:58:54.022904+0000","last_peered":"2024-01-20T08:58:54.022904+0000","last_clean":"2024-01-20T08:58:54.022904+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:58:54.022904+0000","last_undegraded":"2024-01-20T08:58:54.022904+0000","last_fullsized":"2024-01-20T08:58:54.022904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.352940+0000","last_change":"2024-01-20T08:31:08.624784+0000","last_active":"2024-01-20T08:58:41.352940+0000","last_peered":"2024-01-20T08:58:41.352940+0000","last_clean":"2024-01-20T08:58:41.352940+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:58:41.352940+0000","last_undegraded":"2024-01-20T08:58:41.352940+0000","last_fullsized":"2024-01-20T08:58:41.352940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:21:17.745346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"27'188","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.886971+0000","last_change":"2024-01-20T08:31:16.177782+0000","last_active":"2024-01-20T08:58:24.886971+0000","last_peered":"2024-01-20T08:58:24.886971+0000","last_clean":"2024-01-20T08:58:24.886971+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:58:24.886971+0000","last_undegraded":"2024-01-20T08:58:24.886971+0000","last_fullsized":"2024-01-20T08:58:24.886971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:20.608451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'35","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.845305+0000","last_change":"2024-01-20T08:31:44.212803+0000","last_active":"2024-01-20T08:58:53.845305+0000","last_peered":"2024-01-20T08:58:53.845305+0000","last_clean":"2024-01-20T08:58:53.845305+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:58:53.845305+0000","last_undegraded":"2024-01-20T08:58:53.845305+0000","last_fullsized":"2024-01-20T08:58:53.845305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:29.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.295236+0000","last_change":"2024-01-20T08:31:08.594370+0000","last_active":"2024-01-20T08:58:39.295236+0000","last_peered":"2024-01-20T08:58:39.295236+0000","last_clean":"2024-01-20T08:58:39.295236+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:58:39.295236+0000","last_undegraded":"2024-01-20T08:58:39.295236+0000","last_fullsized":"2024-01-20T08:58:39.295236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:46:25.504323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"27'145","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.862293+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:58:24.862293+0000","last_peered":"2024-01-20T08:58:24.862293+0000","last_clean":"2024-01-20T08:58:24.862293+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:58:24.862293+0000","last_undegraded":"2024-01-20T08:58:24.862293+0000","last_fullsized":"2024-01-20T08:58:24.862293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:37.334265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'21","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895345+0000","last_change":"2024-01-20T08:31:10.195197+0000","last_active":"2024-01-20T08:58:53.895345+0000","last_peered":"2024-01-20T08:58:53.895345+0000","last_clean":"2024-01-20T08:58:53.895345+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:58:53.895345+0000","last_undegraded":"2024-01-20T08:58:53.895345+0000","last_fullsized":"2024-01-20T08:58:53.895345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'3","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:33.102678+0000","last_change":"2024-01-20T08:31:08.594805+0000","last_active":"2024-01-20T08:58:33.102678+0000","last_peered":"2024-01-20T08:58:33.102678+0000","last_clean":"2024-01-20T08:58:33.102678+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:58:33.102678+0000","last_undegraded":"2024-01-20T08:58:33.102678+0000","last_fullsized":"2024-01-20T08:58:33.102678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"27'160","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.867693+0000","last_change":"2024-01-20T08:31:16.178580+0000","last_active":"2024-01-20T08:58:25.867693+0000","last_peered":"2024-01-20T08:58:25.867693+0000","last_clean":"2024-01-20T08:58:25.867693+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:58:25.867693+0000","last_undegraded":"2024-01-20T08:58:25.867693+0000","last_fullsized":"2024-01-20T08:58:25.867693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":10942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"27'35","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.882452+0000","last_change":"2024-01-20T08:31:10.235429+0000","last_active":"2024-01-20T08:58:53.882452+0000","last_peered":"2024-01-20T08:58:53.882452+0000","last_clean":"2024-01-20T08:58:53.882452+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:58:53.882452+0000","last_undegraded":"2024-01-20T08:58:53.882452+0000","last_fullsized":"2024-01-20T08:58:53.882452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:49.791051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'21","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:36.115658+0000","last_change":"2024-01-20T08:31:08.592381+0000","last_active":"2024-01-20T08:58:36.115658+0000","last_peered":"2024-01-20T08:58:36.115658+0000","last_clean":"2024-01-20T08:58:36.115658+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:58:36.115658+0000","last_undegraded":"2024-01-20T08:58:36.115658+0000","last_fullsized":"2024-01-20T08:58:36.115658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:49.072049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"27'142","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.848076+0000","last_change":"2024-01-20T08:31:16.174197+0000","last_active":"2024-01-20T08:58:24.848076+0000","last_peered":"2024-01-20T08:58:24.848076+0000","last_clean":"2024-01-20T08:58:24.848076+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:58:24.848076+0000","last_undegraded":"2024-01-20T08:58:24.848076+0000","last_fullsized":"2024-01-20T08:58:24.848076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:09.516428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'29","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881702+0000","last_change":"2024-01-20T08:31:10.277113+0000","last_active":"2024-01-20T08:58:53.881702+0000","last_peered":"2024-01-20T08:58:53.881702+0000","last_clean":"2024-01-20T08:58:53.881702+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:58:53.881702+0000","last_undegraded":"2024-01-20T08:58:53.881702+0000","last_fullsized":"2024-01-20T08:58:53.881702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"27'13","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.790415+0000","last_change":"2024-01-20T08:59:06.790415+0000","last_active":"2024-01-20T08:59:06.790415+0000","last_peered":"2024-01-20T08:59:06.790415+0000","last_clean":"2024-01-20T08:59:06.790415+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:59:06.790415+0000","last_undegraded":"2024-01-20T08:59:06.790415+0000","last_fullsized":"2024-01-20T08:59:06.790415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-01-20T08:59:06.790291+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-01-20T08:59:06.790291+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.790291+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:57.202825+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1169,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"27'133","reported_seq":159,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.549367+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:58:22.549367+0000","last_peered":"2024-01-20T08:58:22.549367+0000","last_clean":"2024-01-20T08:58:22.549367+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:58:22.549367+0000","last_undegraded":"2024-01-20T08:58:22.549367+0000","last_fullsized":"2024-01-20T08:58:22.549367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:43:50.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'48","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875228+0000","last_change":"2024-01-20T08:31:10.235860+0000","last_active":"2024-01-20T08:58:53.875228+0000","last_peered":"2024-01-20T08:58:53.875228+0000","last_clean":"2024-01-20T08:58:53.875228+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:58:53.875228+0000","last_undegraded":"2024-01-20T08:58:53.875228+0000","last_fullsized":"2024-01-20T08:58:53.875228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:52:00.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.531270+0000","last_change":"2024-01-20T08:31:08.584105+0000","last_active":"2024-01-20T08:58:45.531270+0000","last_peered":"2024-01-20T08:58:45.531270+0000","last_clean":"2024-01-20T08:58:45.531270+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:58:45.531270+0000","last_undegraded":"2024-01-20T08:58:45.531270+0000","last_fullsized":"2024-01-20T08:58:45.531270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:34:07.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"27'116","reported_seq":142,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.966157+0000","last_change":"2024-01-20T08:31:16.155065+0000","last_active":"2024-01-20T08:58:23.966157+0000","last_peered":"2024-01-20T08:58:23.966157+0000","last_clean":"2024-01-20T08:58:23.966157+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:58:23.966157+0000","last_undegraded":"2024-01-20T08:58:23.966157+0000","last_fullsized":"2024-01-20T08:58:23.966157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:11.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"27'40","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.803393+0000","last_change":"2024-01-20T08:31:10.285074+0000","last_active":"2024-01-20T08:58:53.803393+0000","last_peered":"2024-01-20T08:58:53.803393+0000","last_clean":"2024-01-20T08:58:53.803393+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:58:53.803393+0000","last_undegraded":"2024-01-20T08:58:53.803393+0000","last_fullsized":"2024-01-20T08:58:53.803393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:39:08.816037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"27'111","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.634588+0000","last_change":"2024-01-20T08:31:08.585617+0000","last_active":"2024-01-20T08:58:53.634588+0000","last_peered":"2024-01-20T08:58:53.634588+0000","last_clean":"2024-01-20T08:58:53.634588+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:58:53.634588+0000","last_undegraded":"2024-01-20T08:58:53.634588+0000","last_fullsized":"2024-01-20T08:58:53.634588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:57:00.515395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069576,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"27'126","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.892559+0000","last_change":"2024-01-20T08:31:16.160875+0000","last_active":"2024-01-20T08:58:23.892559+0000","last_peered":"2024-01-20T08:58:23.892559+0000","last_clean":"2024-01-20T08:58:23.892559+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:58:23.892559+0000","last_undegraded":"2024-01-20T08:58:23.892559+0000","last_fullsized":"2024-01-20T08:58:23.892559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:02:17.760516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'36","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.018998+0000","last_change":"2024-01-20T08:31:10.287917+0000","last_active":"2024-01-20T08:58:54.018998+0000","last_peered":"2024-01-20T08:58:54.018998+0000","last_clean":"2024-01-20T08:58:54.018998+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:58:54.018998+0000","last_undegraded":"2024-01-20T08:58:54.018998+0000","last_fullsized":"2024-01-20T08:58:54.018998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'838","reported_seq":877,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.086905+0000","last_change":"2024-01-20T08:59:06.086932+0000","last_active":"2024-01-20T08:59:06.086905+0000","last_peered":"2024-01-20T08:59:06.086905+0000","last_clean":"2024-01-20T08:59:06.086905+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:59:06.086905+0000","last_undegraded":"2024-01-20T08:59:06.086905+0000","last_fullsized":"2024-01-20T08:59:06.086905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'838","last_scrub_stamp":"2024-01-20T08:59:06.086846+0000","last_deep_scrub":"27'838","last_deep_scrub_stamp":"2024-01-20T08:59:06.086846+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.086846+0000","objects_scrubbed":6,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:03:38.281381+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"27'135","reported_seq":161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.466040+0000","last_change":"2024-01-20T08:31:16.161393+0000","last_active":"2024-01-20T08:58:22.466040+0000","last_peered":"2024-01-20T08:58:22.466040+0000","last_clean":"2024-01-20T08:58:22.466040+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:58:22.466040+0000","last_undegraded":"2024-01-20T08:58:22.466040+0000","last_fullsized":"2024-01-20T08:58:22.466040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T08:40:10.407032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"27'31","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881221+0000","last_change":"2024-01-20T08:31:10.235729+0000","last_active":"2024-01-20T08:58:53.881221+0000","last_peered":"2024-01-20T08:58:53.881221+0000","last_clean":"2024-01-20T08:58:53.881221+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:58:53.881221+0000","last_undegraded":"2024-01-20T08:58:53.881221+0000","last_fullsized":"2024-01-20T08:58:53.881221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:17:17.386140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"27'652","reported_seq":682,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:47.580377+0000","last_change":"2024-01-20T08:31:08.591409+0000","last_active":"2024-01-20T08:58:47.580377+0000","last_peered":"2024-01-20T08:58:47.580377+0000","last_clean":"2024-01-20T08:58:47.580377+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:58:47.580377+0000","last_undegraded":"2024-01-20T08:58:47.580377+0000","last_fullsized":"2024-01-20T08:58:47.580377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:37:38.281846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"27'155","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.874401+0000","last_change":"2024-01-20T08:31:16.179705+0000","last_active":"2024-01-20T08:58:24.874401+0000","last_peered":"2024-01-20T08:58:24.874401+0000","last_clean":"2024-01-20T08:58:24.874401+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:58:24.874401+0000","last_undegraded":"2024-01-20T08:58:24.874401+0000","last_fullsized":"2024-01-20T08:58:24.874401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":23255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.833806+0000","last_change":"2024-01-20T08:31:10.263070+0000","last_active":"2024-01-20T08:58:53.833806+0000","last_peered":"2024-01-20T08:58:53.833806+0000","last_clean":"2024-01-20T08:58:53.833806+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:58:53.833806+0000","last_undegraded":"2024-01-20T08:58:53.833806+0000","last_fullsized":"2024-01-20T08:58:53.833806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:37:04.653251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:44.540317+0000","last_change":"2024-01-20T08:31:08.624850+0000","last_active":"2024-01-20T08:58:44.540317+0000","last_peered":"2024-01-20T08:58:44.540317+0000","last_clean":"2024-01-20T08:58:44.540317+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:58:44.540317+0000","last_undegraded":"2024-01-20T08:58:44.540317+0000","last_fullsized":"2024-01-20T08:58:44.540317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"27'124","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.864733+0000","last_change":"2024-01-20T08:31:16.179562+0000","last_active":"2024-01-20T08:58:24.864733+0000","last_peered":"2024-01-20T08:58:24.864733+0000","last_clean":"2024-01-20T08:58:24.864733+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:58:24.864733+0000","last_undegraded":"2024-01-20T08:58:24.864733+0000","last_fullsized":"2024-01-20T08:58:24.864733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:14:54.342372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"27'42","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895691+0000","last_change":"2024-01-20T08:31:10.234283+0000","last_active":"2024-01-20T08:58:53.895691+0000","last_peered":"2024-01-20T08:58:53.895691+0000","last_clean":"2024-01-20T08:58:53.895691+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:58:53.895691+0000","last_undegraded":"2024-01-20T08:58:53.895691+0000","last_fullsized":"2024-01-20T08:58:53.895691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'14","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.633916+0000","last_change":"2024-01-20T08:31:08.625851+0000","last_active":"2024-01-20T08:58:53.633916+0000","last_peered":"2024-01-20T08:58:53.633916+0000","last_clean":"2024-01-20T08:58:53.633916+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:58:53.633916+0000","last_undegraded":"2024-01-20T08:58:53.633916+0000","last_fullsized":"2024-01-20T08:58:53.633916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"27'155","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:22.554720+0000","last_change":"2024-01-20T08:31:16.155194+0000","last_active":"2024-01-20T08:58:22.554720+0000","last_peered":"2024-01-20T08:58:22.554720+0000","last_clean":"2024-01-20T08:58:22.554720+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:58:22.554720+0000","last_undegraded":"2024-01-20T08:58:22.554720+0000","last_fullsized":"2024-01-20T08:58:22.554720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:43:08.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"27'130","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.958134+0000","last_change":"2024-01-20T08:31:16.174738+0000","last_active":"2024-01-20T08:58:23.958134+0000","last_peered":"2024-01-20T08:58:23.958134+0000","last_clean":"2024-01-20T08:58:23.958134+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:58:23.958134+0000","last_undegraded":"2024-01-20T08:58:23.958134+0000","last_fullsized":"2024-01-20T08:58:23.958134+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:33:05.847035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":8002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.932256+0000","last_change":"2024-01-20T08:31:10.277117+0000","last_active":"2024-01-20T08:58:53.932256+0000","last_peered":"2024-01-20T08:58:53.932256+0000","last_clean":"2024-01-20T08:58:53.932256+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:58:53.932256+0000","last_undegraded":"2024-01-20T08:58:53.932256+0000","last_fullsized":"2024-01-20T08:58:53.932256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:41.360632+0000","last_change":"2024-01-20T08:31:08.624147+0000","last_active":"2024-01-20T08:58:41.360632+0000","last_peered":"2024-01-20T08:58:41.360632+0000","last_clean":"2024-01-20T08:58:41.360632+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:58:41.360632+0000","last_undegraded":"2024-01-20T08:58:41.360632+0000","last_fullsized":"2024-01-20T08:58:41.360632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:41:10.231471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"27'143","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.710016+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:58:24.710016+0000","last_peered":"2024-01-20T08:58:24.710016+0000","last_clean":"2024-01-20T08:58:24.710016+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:58:24.710016+0000","last_undegraded":"2024-01-20T08:58:24.710016+0000","last_fullsized":"2024-01-20T08:58:24.710016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:16:32.778128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":18955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'41","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.841394+0000","last_change":"2024-01-20T08:31:10.287293+0000","last_active":"2024-01-20T08:58:53.841394+0000","last_peered":"2024-01-20T08:58:53.841394+0000","last_clean":"2024-01-20T08:58:53.841394+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:58:53.841394+0000","last_undegraded":"2024-01-20T08:58:53.841394+0000","last_fullsized":"2024-01-20T08:58:53.841394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:09:33.656380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.294844+0000","last_change":"2024-01-20T08:31:08.557295+0000","last_active":"2024-01-20T08:58:39.294844+0000","last_peered":"2024-01-20T08:58:39.294844+0000","last_clean":"2024-01-20T08:58:39.294844+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:58:39.294844+0000","last_undegraded":"2024-01-20T08:58:39.294844+0000","last_fullsized":"2024-01-20T08:58:39.294844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:37:05.406471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"27'675","reported_seq":705,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:39.711307+0000","last_change":"2024-01-20T08:31:08.584608+0000","last_active":"2024-01-20T08:58:39.711307+0000","last_peered":"2024-01-20T08:58:39.711307+0000","last_clean":"2024-01-20T08:58:39.711307+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:58:39.711307+0000","last_undegraded":"2024-01-20T08:58:39.711307+0000","last_fullsized":"2024-01-20T08:58:39.711307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:47:46.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.929004+0000","last_change":"2024-01-20T08:31:10.195181+0000","last_active":"2024-01-20T08:58:53.929004+0000","last_peered":"2024-01-20T08:58:53.929004+0000","last_clean":"2024-01-20T08:58:53.929004+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:58:53.929004+0000","last_undegraded":"2024-01-20T08:58:53.929004+0000","last_fullsized":"2024-01-20T08:58:53.929004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"27'89","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.877290+0000","last_change":"2024-01-20T08:31:16.161985+0000","last_active":"2024-01-20T08:58:23.877290+0000","last_peered":"2024-01-20T08:58:23.877290+0000","last_clean":"2024-01-20T08:58:23.877290+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:58:23.877290+0000","last_undegraded":"2024-01-20T08:58:23.877290+0000","last_fullsized":"2024-01-20T08:58:23.877290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:28:32.446627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":9333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'689","reported_seq":719,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:45.569769+0000","last_change":"2024-01-20T08:31:08.595941+0000","last_active":"2024-01-20T08:58:45.569769+0000","last_peered":"2024-01-20T08:58:45.569769+0000","last_clean":"2024-01-20T08:58:45.569769+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:58:45.569769+0000","last_undegraded":"2024-01-20T08:58:45.569769+0000","last_fullsized":"2024-01-20T08:58:45.569769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:07.443764+0000","last_clean_scrub_stamp":"2024-01-20T08:31:07.443764+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:19:29.483359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"27'33","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888717+0000","last_change":"2024-01-20T08:31:10.278524+0000","last_active":"2024-01-20T08:58:53.888717+0000","last_peered":"2024-01-20T08:58:53.888717+0000","last_clean":"2024-01-20T08:58:53.888717+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:58:53.888717+0000","last_undegraded":"2024-01-20T08:58:53.888717+0000","last_fullsized":"2024-01-20T08:58:53.888717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"27'147","reported_seq":173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:24.840471+0000","last_change":"2024-01-20T08:31:16.179459+0000","last_active":"2024-01-20T08:58:24.840471+0000","last_peered":"2024-01-20T08:58:24.840471+0000","last_clean":"2024-01-20T08:58:24.840471+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:58:24.840471+0000","last_undegraded":"2024-01-20T08:58:24.840471+0000","last_fullsized":"2024-01-20T08:58:24.840471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:16:38.387611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":10430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"27'50","reported_seq":82,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892250+0000","last_change":"2024-01-20T08:31:10.287352+0000","last_active":"2024-01-20T08:58:53.892250+0000","last_peered":"2024-01-20T08:58:53.892250+0000","last_clean":"2024-01-20T08:58:53.892250+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:58:53.892250+0000","last_undegraded":"2024-01-20T08:58:53.892250+0000","last_fullsized":"2024-01-20T08:58:53.892250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"27'50","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843335+0000","last_change":"2024-01-20T08:31:10.278902+0000","last_active":"2024-01-20T08:58:53.843335+0000","last_peered":"2024-01-20T08:58:53.843335+0000","last_clean":"2024-01-20T08:58:53.843335+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:58:53.843335+0000","last_undegraded":"2024-01-20T08:58:53.843335+0000","last_fullsized":"2024-01-20T08:58:53.843335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:13:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"27'44","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.875423+0000","last_change":"2024-01-20T08:31:10.260541+0000","last_active":"2024-01-20T08:58:53.875423+0000","last_peered":"2024-01-20T08:58:53.875423+0000","last_clean":"2024-01-20T08:58:53.875423+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:58:53.875423+0000","last_undegraded":"2024-01-20T08:58:53.875423+0000","last_fullsized":"2024-01-20T08:58:53.875423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:03:04.202796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.839038+0000","last_change":"2024-01-20T08:31:10.260201+0000","last_active":"2024-01-20T08:58:53.839038+0000","last_peered":"2024-01-20T08:58:53.839038+0000","last_clean":"2024-01-20T08:58:53.839038+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:58:53.839038+0000","last_undegraded":"2024-01-20T08:58:53.839038+0000","last_fullsized":"2024-01-20T08:58:53.839038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:12:38.983091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"27'27","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.807055+0000","last_change":"2024-01-20T08:31:10.287570+0000","last_active":"2024-01-20T08:58:53.807055+0000","last_peered":"2024-01-20T08:58:53.807055+0000","last_clean":"2024-01-20T08:58:53.807055+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:58:53.807055+0000","last_undegraded":"2024-01-20T08:58:53.807055+0000","last_fullsized":"2024-01-20T08:58:53.807055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:17:58.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'31","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.802128+0000","last_change":"2024-01-20T08:31:10.284942+0000","last_active":"2024-01-20T08:58:53.802128+0000","last_peered":"2024-01-20T08:58:53.802128+0000","last_clean":"2024-01-20T08:58:53.802128+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:58:53.802128+0000","last_undegraded":"2024-01-20T08:58:53.802128+0000","last_fullsized":"2024-01-20T08:58:53.802128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:30:39.109156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"27'44","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.895544+0000","last_change":"2024-01-20T08:31:10.234268+0000","last_active":"2024-01-20T08:58:53.895544+0000","last_peered":"2024-01-20T08:58:53.895544+0000","last_clean":"2024-01-20T08:58:53.895544+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:58:53.895544+0000","last_undegraded":"2024-01-20T08:58:53.895544+0000","last_fullsized":"2024-01-20T08:58:53.895544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.808904+0000","last_change":"2024-01-20T08:31:10.278984+0000","last_active":"2024-01-20T08:58:53.808904+0000","last_peered":"2024-01-20T08:58:53.808904+0000","last_clean":"2024-01-20T08:58:53.808904+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:58:53.808904+0000","last_undegraded":"2024-01-20T08:58:53.808904+0000","last_fullsized":"2024-01-20T08:58:53.808904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:48:21.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.871511+0000","last_change":"2024-01-20T08:31:10.263812+0000","last_active":"2024-01-20T08:58:53.871511+0000","last_peered":"2024-01-20T08:58:53.871511+0000","last_clean":"2024-01-20T08:58:53.871511+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:58:53.871511+0000","last_undegraded":"2024-01-20T08:58:53.871511+0000","last_fullsized":"2024-01-20T08:58:53.871511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:01:32.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"27'42","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.837960+0000","last_change":"2024-01-20T08:31:10.259917+0000","last_active":"2024-01-20T08:58:53.837960+0000","last_peered":"2024-01-20T08:58:53.837960+0000","last_clean":"2024-01-20T08:58:53.837960+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:58:53.837960+0000","last_undegraded":"2024-01-20T08:58:53.837960+0000","last_fullsized":"2024-01-20T08:58:53.837960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"27'38","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.888783+0000","last_change":"2024-01-20T08:31:10.284766+0000","last_active":"2024-01-20T08:58:53.888783+0000","last_peered":"2024-01-20T08:58:53.888783+0000","last_clean":"2024-01-20T08:58:53.888783+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:58:53.888783+0000","last_undegraded":"2024-01-20T08:58:53.888783+0000","last_fullsized":"2024-01-20T08:58:53.888783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'47","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.928357+0000","last_change":"2024-01-20T08:31:10.191326+0000","last_active":"2024-01-20T08:58:53.928357+0000","last_peered":"2024-01-20T08:58:53.928357+0000","last_clean":"2024-01-20T08:58:53.928357+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:58:53.928357+0000","last_undegraded":"2024-01-20T08:58:53.928357+0000","last_fullsized":"2024-01-20T08:58:53.928357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:11:09.826287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"27'41","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023663+0000","last_change":"2024-01-20T08:31:10.277818+0000","last_active":"2024-01-20T08:58:54.023663+0000","last_peered":"2024-01-20T08:58:54.023663+0000","last_clean":"2024-01-20T08:58:54.023663+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:58:54.023663+0000","last_undegraded":"2024-01-20T08:58:54.023663+0000","last_fullsized":"2024-01-20T08:58:54.023663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"27'34","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.878349+0000","last_change":"2024-01-20T08:31:10.236873+0000","last_active":"2024-01-20T08:58:53.878349+0000","last_peered":"2024-01-20T08:58:53.878349+0000","last_clean":"2024-01-20T08:58:53.878349+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:58:53.878349+0000","last_undegraded":"2024-01-20T08:58:53.878349+0000","last_fullsized":"2024-01-20T08:58:53.878349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:00:26.640030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"27'45","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.846302+0000","last_change":"2024-01-20T08:31:10.236453+0000","last_active":"2024-01-20T08:58:53.846302+0000","last_peered":"2024-01-20T08:58:53.846302+0000","last_clean":"2024-01-20T08:58:53.846302+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:58:53.846302+0000","last_undegraded":"2024-01-20T08:58:53.846302+0000","last_fullsized":"2024-01-20T08:58:53.846302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:00:12.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"27'38","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.870937+0000","last_change":"2024-01-20T08:31:10.236531+0000","last_active":"2024-01-20T08:58:53.870937+0000","last_peered":"2024-01-20T08:58:53.870937+0000","last_clean":"2024-01-20T08:58:53.870937+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:58:53.870937+0000","last_undegraded":"2024-01-20T08:58:53.870937+0000","last_fullsized":"2024-01-20T08:58:53.870937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:16:02.866034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"27'42","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.881740+0000","last_change":"2024-01-20T08:31:10.235633+0000","last_active":"2024-01-20T08:58:53.881740+0000","last_peered":"2024-01-20T08:58:53.881740+0000","last_clean":"2024-01-20T08:58:53.881740+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:58:53.881740+0000","last_undegraded":"2024-01-20T08:58:53.881740+0000","last_fullsized":"2024-01-20T08:58:53.881740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:30:36.678660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"27'39","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.834894+0000","last_change":"2024-01-20T08:31:10.259767+0000","last_active":"2024-01-20T08:58:53.834894+0000","last_peered":"2024-01-20T08:58:53.834894+0000","last_clean":"2024-01-20T08:58:53.834894+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:58:53.834894+0000","last_undegraded":"2024-01-20T08:58:53.834894+0000","last_fullsized":"2024-01-20T08:58:53.834894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:23:19.809953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"27'36","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.804360+0000","last_change":"2024-01-20T08:31:10.287286+0000","last_active":"2024-01-20T08:58:53.804360+0000","last_peered":"2024-01-20T08:58:53.804360+0000","last_clean":"2024-01-20T08:58:53.804360+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:58:53.804360+0000","last_undegraded":"2024-01-20T08:58:53.804360+0000","last_fullsized":"2024-01-20T08:58:53.804360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:28:51.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"27'37","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806862+0000","last_change":"2024-01-20T08:31:10.288288+0000","last_active":"2024-01-20T08:58:53.806862+0000","last_peered":"2024-01-20T08:58:53.806862+0000","last_clean":"2024-01-20T08:58:53.806862+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:58:53.806862+0000","last_undegraded":"2024-01-20T08:58:53.806862+0000","last_fullsized":"2024-01-20T08:58:53.806862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:07:19.530067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'28","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023508+0000","last_change":"2024-01-20T08:31:10.278856+0000","last_active":"2024-01-20T08:58:54.023508+0000","last_peered":"2024-01-20T08:58:54.023508+0000","last_clean":"2024-01-20T08:58:54.023508+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:58:54.023508+0000","last_undegraded":"2024-01-20T08:58:54.023508+0000","last_fullsized":"2024-01-20T08:58:54.023508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"27'31","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023107+0000","last_change":"2024-01-20T08:31:10.234607+0000","last_active":"2024-01-20T08:58:54.023107+0000","last_peered":"2024-01-20T08:58:54.023107+0000","last_clean":"2024-01-20T08:58:54.023107+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:58:54.023107+0000","last_undegraded":"2024-01-20T08:58:54.023107+0000","last_fullsized":"2024-01-20T08:58:54.023107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"27'48","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.023844+0000","last_change":"2024-01-20T08:31:10.195892+0000","last_active":"2024-01-20T08:58:54.023844+0000","last_peered":"2024-01-20T08:58:54.023844+0000","last_clean":"2024-01-20T08:58:54.023844+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:58:54.023844+0000","last_undegraded":"2024-01-20T08:58:54.023844+0000","last_fullsized":"2024-01-20T08:58:54.023844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"27'37","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.889830+0000","last_change":"2024-01-20T08:31:10.281270+0000","last_active":"2024-01-20T08:58:53.889830+0000","last_peered":"2024-01-20T08:58:53.889830+0000","last_clean":"2024-01-20T08:58:53.889830+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:58:53.889830+0000","last_undegraded":"2024-01-20T08:58:53.889830+0000","last_fullsized":"2024-01-20T08:58:53.889830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.843441+0000","last_change":"2024-01-20T08:31:10.263686+0000","last_active":"2024-01-20T08:58:53.843441+0000","last_peered":"2024-01-20T08:58:53.843441+0000","last_clean":"2024-01-20T08:58:53.843441+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:58:53.843441+0000","last_undegraded":"2024-01-20T08:58:53.843441+0000","last_fullsized":"2024-01-20T08:58:53.843441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:47:53.947876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'36","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:05.751587+0000","last_change":"2024-01-20T08:59:05.751608+0000","last_active":"2024-01-20T08:59:05.751587+0000","last_peered":"2024-01-20T08:59:05.751587+0000","last_clean":"2024-01-20T08:59:05.751587+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:59:05.751587+0000","last_undegraded":"2024-01-20T08:59:05.751587+0000","last_fullsized":"2024-01-20T08:59:05.751587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:05.751528+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:05.751528+0000","last_clean_scrub_stamp":"2024-01-20T08:59:05.751528+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:40:23.102642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"27'37","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892387+0000","last_change":"2024-01-20T08:31:10.192533+0000","last_active":"2024-01-20T08:58:53.892387+0000","last_peered":"2024-01-20T08:58:53.892387+0000","last_clean":"2024-01-20T08:58:53.892387+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:58:53.892387+0000","last_undegraded":"2024-01-20T08:58:53.892387+0000","last_fullsized":"2024-01-20T08:58:53.892387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"27'30","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:54.019490+0000","last_change":"2024-01-20T08:31:10.263205+0000","last_active":"2024-01-20T08:58:54.019490+0000","last_peered":"2024-01-20T08:58:54.019490+0000","last_clean":"2024-01-20T08:58:54.019490+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:58:54.019490+0000","last_undegraded":"2024-01-20T08:58:54.019490+0000","last_fullsized":"2024-01-20T08:58:54.019490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'25","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.892252+0000","last_change":"2024-01-20T08:31:10.191136+0000","last_active":"2024-01-20T08:58:53.892252+0000","last_peered":"2024-01-20T08:58:53.892252+0000","last_clean":"2024-01-20T08:58:53.892252+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:58:53.892252+0000","last_undegraded":"2024-01-20T08:58:53.892252+0000","last_fullsized":"2024-01-20T08:58:53.892252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'41","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.842487+0000","last_change":"2024-01-20T08:31:10.262527+0000","last_active":"2024-01-20T08:58:53.842487+0000","last_peered":"2024-01-20T08:58:53.842487+0000","last_clean":"2024-01-20T08:58:53.842487+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:58:53.842487+0000","last_undegraded":"2024-01-20T08:58:53.842487+0000","last_fullsized":"2024-01-20T08:58:53.842487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:44.738708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"27'33","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:53.806164+0000","last_change":"2024-01-20T08:31:10.284854+0000","last_active":"2024-01-20T08:58:53.806164+0000","last_peered":"2024-01-20T08:58:53.806164+0000","last_clean":"2024-01-20T08:58:53.806164+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:58:53.806164+0000","last_undegraded":"2024-01-20T08:58:53.806164+0000","last_fullsized":"2024-01-20T08:58:53.806164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:08.472136+0000","last_clean_scrub_stamp":"2024-01-20T08:31:08.472136+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:59:01.673470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"27'26","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:04.749273+0000","last_change":"2024-01-20T08:59:04.749310+0000","last_active":"2024-01-20T08:59:04.749273+0000","last_peered":"2024-01-20T08:59:04.749273+0000","last_clean":"2024-01-20T08:59:04.749273+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:59:04.749273+0000","last_undegraded":"2024-01-20T08:59:04.749273+0000","last_fullsized":"2024-01-20T08:59:04.749273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-01-20T08:59:04.749186+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-01-20T08:59:04.749186+0000","last_clean_scrub_stamp":"2024-01-20T08:59:04.749186+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:15.202067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":308552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4420,"ondisk_log_size":4420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39037201,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":8593,"num_write_kb":43222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78323712,"data_stored":78074402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8123,"ondisk_log_size":8123,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509786,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39732,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332108,"statfs":{"total":96636764160,"available":96596078592,"internally_reserved":0,"allocated":4501504,"data_stored":4315319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46999999999999997,"15min":0.45300000000000001},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.23899999999999999},"max":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.25800000000000001},"max":{"1min":0.60999999999999999,"5min":0.66200000000000003,"15min":0.81999999999999995},"last":0.47399999999999998}]},{"osd":1,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.67900000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28999999999999998},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.67500000000000004},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.70899999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.628,"5min":0.68500000000000005,"15min":0.72699999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.30299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.74199999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44800000000000001,"15min":0.45400000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.23799999999999999},"max":{"1min":0.63500000000000001,"5min":0.67400000000000004,"15min":0.76700000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75800000000000001,"15min":0.876},"last":0.437}]},{"osd":5,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.81000000000000005},"last":0.499},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.79400000000000004},"last":0.39600000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.27600000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.90900000000000003},"last":0.38},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"last":0.41799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509786,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67036,"kb_used_data":13652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94304804,"statfs":{"total":96636764160,"available":96568119296,"internally_reserved":0,"allocated":13979648,"data_stored":13773597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.29099999999999998},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17199999999999999},"max":{"1min":0.41899999999999998,"5min":0.441,"15min":0.72599999999999998},"last":0.316},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.19500000000000001,"15min":0.19400000000000001},"max":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.58799999999999997},"last":0.246}]},{"osd":2,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.503,"15min":0.61799999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.216,"15min":0.21099999999999999},"max":{"1min":0.39500000000000002,"5min":0.51000000000000001,"15min":0.79600000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.69499999999999995},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.72599999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56499999999999995}]},{"osd":5,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80100000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.77600000000000002,"15min":0.91400000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.57799999999999996}]},{"osd":7,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.78500000000000003},"last":0.55100000000000005}]}]},{"osd":5,"up_from":14,"seq":60129542488,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70924,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300916,"statfs":{"total":96636764160,"available":96564137984,"internally_reserved":0,"allocated":13701120,"data_stored":13502280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"max":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.69599999999999995},"last":0.434},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72099999999999997},"last":0.36299999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.66000000000000003},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.72899999999999998},"last":0.378},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.37}]},{"osd":3,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.76700000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.80900000000000005},"last":0.39500000000000002}]},{"osd":4,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44,"15min":0.439},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48399999999999999,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.26500000000000001},"max":{"1min":0.56799999999999995,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.28699999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.252},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.81599999999999995},"last":0.23000000000000001}]},{"osd":7,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.80300000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59399999999999997,"5min":0.76200000000000001,"15min":0.85799999999999998},"last":0.35099999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542487,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66068,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305772,"statfs":{"total":96636764160,"available":96569110528,"internally_reserved":0,"allocated":8728576,"data_stored":8508591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.61299999999999999,"15min":0.70199999999999996},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.46899999999999997}]},{"osd":1,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.629,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.31,"15min":0.26500000000000001},"max":{"1min":0.51400000000000001,"5min":0.69599999999999995,"15min":0.73299999999999998},"last":0.44},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.315,"5min":0.31,"15min":0.28599999999999998},"max":{"1min":0.54700000000000004,"5min":0.622,"15min":0.66700000000000004},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57799999999999996,"5min":0.71899999999999997,"15min":0.73999999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.254},"max":{"1min":0.61899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.30599999999999999}]},{"osd":6,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.59099999999999997,"5min":0.75700000000000001,"15min":0.78400000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":0.75600000000000001},"last":0.38200000000000001}]},{"osd":7,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66000000000000003,"5min":0.78300000000000003,"15min":0.79100000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":0.71699999999999997,"15min":0.79000000000000004},"last":0.48999999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542488,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43824,"kb_used_data":8488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328016,"statfs":{"total":96636764160,"available":96591888384,"internally_reserved":0,"allocated":8691712,"data_stored":8506821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.45800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29999999999999999,"15min":0.307},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":0.38100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.24399999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.502,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.27900000000000003}]},{"osd":3,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33000000000000002,"15min":0.33200000000000002},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.378,"5min":0.55900000000000005,"15min":0.60799999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36699999999999999}]},{"osd":4,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.71399999999999997,"15min":0.82099999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.40600000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.436},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.86699999999999999},"last":0.379}]},{"osd":6,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.80500000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.69999999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"last":0.46400000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.76000000000000001,"15min":0.79300000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.36399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85299999999999998},"last":0.47799999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542489,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9289728,"data_stored":9097863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30099999999999999,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.55700000000000005,"15min":0.59499999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.32000000000000001,"15min":0.31},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20000000000000001},"max":{"1min":0.40600000000000003,"5min":0.45500000000000002,"15min":0.54400000000000004},"last":0.246}]},{"osd":2,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.308,"15min":0.314},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.35399999999999998,"5min":0.47999999999999998,"15min":0.49199999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":0.45800000000000002,"5min":0.624,"15min":0.624},"last":0.37}]},{"osd":3,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.59399999999999997,"15min":0.63},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.44700000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.38300000000000001}]},{"osd":4,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.27900000000000003},"max":{"1min":0.80700000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56200000000000006}]},{"osd":5,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.876},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":0.873,"15min":0.873},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54900000000000004}]}]},{"osd":7,"up_from":13,"seq":55834575191,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63212,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308628,"statfs":{"total":96636764160,"available":96572035072,"internally_reserved":0,"allocated":5804032,"data_stored":5592459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.68999999999999995},"last":0.52700000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.54200000000000004,"5min":0.65100000000000002,"15min":0.73499999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.37,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.78000000000000003},"last":0.499}]},{"osd":4,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23000000000000001},"max":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.75800000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26600000000000001},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.79900000000000004},"last":0.28199999999999997}]},{"osd":5,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"last":0.378},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.76400000000000001,"15min":0.77800000000000002},"last":0.32700000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.36099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575191,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61772,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310068,"statfs":{"total":96636764160,"available":96573509632,"internally_reserved":0,"allocated":17108992,"data_stored":16897992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.505,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.31,"15min":0.30399999999999999},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.41699999999999998,"5min":0.47599999999999998,"15min":0.51200000000000001},"last":0.32900000000000001}]},{"osd":1,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47199999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.42299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.30499999999999999}]},{"osd":2,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44700000000000001,"5min":0.625,"15min":0.625},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.373,"15min":0.371},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.434,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.78500000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.432},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.71399999999999997,"5min":0.80000000000000004,"15min":0.85499999999999998},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.76100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45300000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.34000000000000002},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.442}]},{"osd":7,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.41899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13692928,"data_stored":13656124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8409088,"data_stored":8390022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":8389348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16822272,"data_stored":16780519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8441856,"data_stored":8391118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12820480,"data_stored":12794439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4214784,"data_stored":4197846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":5474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:59:10.167 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.167 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.167 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.167 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.168 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.169 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.170 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.171 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.172 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.173 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.174 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.175 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-01-20T08:31:07.443764+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.176 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.177 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.179 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.180 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.181 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.181 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.181 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.181 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-01-20T08:31:08.472136+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=8, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:10.181 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-20T08:59:16.504 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:59:16.507 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:59:30.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:59:30.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.286+0000 7fd5d0f3a640 1 Processor -- start 2024-01-20T08:59:30.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.286+0000 7fd5d0f3a640 1 -- start start 2024-01-20T08:59:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.287+0000 7fd5d0f3a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd5cc0bff90 0x7fd5cc0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.287+0000 7fd5d0f3a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5cc156ea0 0x7fd5cc1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.287+0000 7fd5d0f3a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5cc071910 con 0x7fd5cc150350 2024-01-20T08:59:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.287+0000 7fd5d0f3a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5cc150730 con 0x7fd5cc0bff90 2024-01-20T08:59:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.287+0000 7fd5d0f3a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5cc071b60 con 0x7fd5cc156ea0 2024-01-20T08:59:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.288+0000 7fd5c9d9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd5cc0bff90 0x7fd5cc0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:30.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.288+0000 7fd5cad9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5cc156ea0 0x7fd5cc1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:30.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.288+0000 7fd5cad9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5cc156ea0 0x7fd5cc1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:53338/0 (socket says 172.21.15.87:53338) 2024-01-20T08:59:30.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.288+0000 7fd5cad9d640 1 -- 172.21.15.87:0/845302602 learned_addr learned my addr 172.21.15.87:0/845302602 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:30.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.290+0000 7fd5cad9d640 1 -- 172.21.15.87:0/845302602 >> v2:172.21.15.87:3300/0 conn(0x7fd5cc0bff90 msgr2=0x7fd5cc0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:30.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.291+0000 7fd5cad9d640 1 --2- 172.21.15.87:0/845302602 >> v2:172.21.15.87:3300/0 conn(0x7fd5cc0bff90 0x7fd5cc0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.291+0000 7fd5cad9d640 1 -- 172.21.15.87:0/845302602 >> v1:172.21.15.87:6789/0 conn(0x7fd5cc150350 legacy=0x7fd5cc0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:30.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.291+0000 7fd5cad9d640 1 -- 172.21.15.87:0/845302602 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc1563c0 con 0x7fd5cc156ea0 2024-01-20T08:59:30.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.292+0000 7fd5cad9d640 1 --2- 172.21.15.87:0/845302602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc156ea0 0x7fd5cc1598a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc06e700 tx=0x7fd5c402e180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f306068ad51ad099 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:30.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.296+0000 7fd5c959a640 1 -- 172.21.15.87:0/845302602 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400b0d0 con 0x7fd5cc156ea0 2024-01-20T08:59:30.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.296+0000 7fd5c959a640 1 -- 172.21.15.87:0/845302602 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c400b230 con 0x7fd5cc156ea0 2024-01-20T08:59:30.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.296+0000 7fd5c959a640 1 -- 172.21.15.87:0/845302602 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4009600 con 0x7fd5cc156ea0 2024-01-20T08:59:30.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/845302602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc156ea0 msgr2=0x7fd5cc1598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/845302602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc156ea0 0x7fd5cc1598a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc06e700 tx=0x7fd5c402e180 comp rx=0 tx=0).stop 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/845302602 shutdown_connections 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/845302602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc156ea0 0x7fd5cc1598a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/845302602 >> v2:172.21.15.87:3300/0 conn(0x7fd5cc0bff90 0x7fd5cc0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/845302602 >> 172.21.15.87:0/845302602 conn(0x7fd5cc06d510 msgr2=0x7fd5cc0663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/845302602 shutdown_connections 2024-01-20T08:59:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.297+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/845302602 wait complete. 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.298+0000 7fd5d0f3a640 1 Processor -- start 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.298+0000 7fd5d0f3a640 1 -- start start 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5d0f3a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 0x7fd5cc1dfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5d0f3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc150350 0x7fd5cc147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5d0f3a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd5cc156ea0 0x7fd5cc155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5d0f3a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc154500 con 0x7fd5cc0bff90 2024-01-20T08:59:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5d0f3a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1431c0 con 0x7fd5cc150350 2024-01-20T08:59:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5d0f3a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc154710 con 0x7fd5cc156ea0 2024-01-20T08:59:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.299+0000 7fd5ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 0x7fd5cc1dfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.300+0000 7fd5c9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc150350 0x7fd5cc147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.300+0000 7fd5cad9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd5cc156ea0 0x7fd5cc155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.300+0000 7fd5ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 0x7fd5cc1dfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44294/0 (socket says 172.21.15.87:44294) 2024-01-20T08:59:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.300+0000 7fd5ca59c640 1 -- 172.21.15.87:0/3537491153 learned_addr learned my addr 172.21.15.87:0/3537491153 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.303+0000 7fd5ca59c640 1 -- 172.21.15.87:0/3537491153 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd5cc156ea0 msgr2=0x7fd5cc155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.303+0000 7fd5ca59c640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd5cc156ea0 0x7fd5cc155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.303+0000 7fd5ca59c640 1 -- 172.21.15.87:0/3537491153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc150350 msgr2=0x7fd5cc147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.303+0000 7fd5ca59c640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc150350 0x7fd5cc147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.303+0000 7fd5ca59c640 1 -- 172.21.15.87:0/3537491153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c400bd40 con 0x7fd5cc0bff90 2024-01-20T08:59:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.305+0000 7fd5ca59c640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 0x7fd5cc1dfbf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000c400 tx=0x7fd5c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.309+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00052d0 con 0x7fd5cc0bff90 2024-01-20T08:59:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.309+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c0005450 con 0x7fd5cc0bff90 2024-01-20T08:59:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.309+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0007260 con 0x7fd5cc0bff90 2024-01-20T08:59:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.309+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc1487a0 con 0x7fd5cc0bff90 2024-01-20T08:59:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.309+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc1efab0 con 0x7fd5cc0bff90 2024-01-20T08:59:30.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.310+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5cc074290 con 0x7fd5cc0bff90 2024-01-20T08:59:30.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.328+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd5c00073c0 con 0x7fd5cc0bff90 2024-01-20T08:59:30.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.329+0000 7fd5baffd640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd5ac06af20 0x7fd5ac06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:30.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.329+0000 7fd5c9d9b640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd5ac06af20 0x7fd5ac06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:30.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.329+0000 7fd5c9d9b640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd5ac06af20 0x7fd5ac06d3e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc14a480 tx=0x7fd5bc009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:30.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.565+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fd5c0007670 con 0x7fd5cc0bff90 2024-01-20T08:59:30.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.565+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd5c0054190 con 0x7fd5cc0bff90 2024-01-20T08:59:30.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.736+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5cc156980 con 0x7fd5ac06af20 2024-01-20T08:59:30.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.745+0000 7fd5baffd640 1 -- 172.21.15.87:0/3537491153 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371753 (secure 0 0 0) 0x7fd5bc011a30 con 0x7fd5ac06af20 2024-01-20T08:59:30.750 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:30.752 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd5ac06af20 msgr2=0x7fd5ac06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd5ac06af20 0x7fd5ac06d3e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc14a480 tx=0x7fd5bc009190 comp rx=0 tx=0).stop 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 msgr2=0x7fd5cc1dfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 0x7fd5cc1dfbf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000c400 tx=0x7fd5c000c8c0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 shutdown_connections 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fd5ac06af20 0x7fd5ac06d3e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd5cc156ea0 0x7fd5cc155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5cc150350 0x7fd5cc147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 --2- 172.21.15.87:0/3537491153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5cc0bff90 0x7fd5cc1dfbf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:30.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 >> 172.21.15.87:0/3537491153 conn(0x7fd5cc06d510 msgr2=0x7fd5cc0c16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:30.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 shutdown_connections 2024-01-20T08:59:30.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:30.749+0000 7fd5d0f3a640 1 -- 172.21.15.87:0/3537491153 wait complete. 2024-01-20T08:59:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":881,"stamp":"2024-01-20T08:59:30.659853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39627569,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":1083,"num_read_kb":1128,"num_write":12670,"num_write_kb":354190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15074,"ondisk_log_size":15074,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461136,"kb_used_data":79888,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754513584,"statfs":{"total":773094113280,"available":772621910016,"internally_reserved":0,"allocated":81805312,"data_stored":80194922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390319664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002946"},"pg_stats":[{"pgid":"2.1f","version":"27'616","reported_seq":657,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.707135+0000","last_change":"2024-01-20T08:59:11.707161+0000","last_active":"2024-01-20T08:59:11.707135+0000","last_peered":"2024-01-20T08:59:11.707135+0000","last_clean":"2024-01-20T08:59:11.707135+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:59:11.707135+0000","last_undegraded":"2024-01-20T08:59:11.707135+0000","last_fullsized":"2024-01-20T08:59:11.707135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'616","last_scrub_stamp":"2024-01-20T08:59:11.707082+0000","last_deep_scrub":"27'616","last_deep_scrub_stamp":"2024-01-20T08:59:11.707082+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.707082+0000","objects_scrubbed":7,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:25.740999+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"27'27","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.005143+0000","last_change":"2024-01-20T08:59:18.005164+0000","last_active":"2024-01-20T08:59:18.005143+0000","last_peered":"2024-01-20T08:59:18.005143+0000","last_clean":"2024-01-20T08:59:18.005143+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:59:18.005143+0000","last_undegraded":"2024-01-20T08:59:18.005143+0000","last_fullsized":"2024-01-20T08:59:18.005143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-01-20T08:59:18.005080+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-01-20T08:59:18.005080+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.005080+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:43:18.802602+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"27'162","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.735639+0000","last_change":"2024-01-20T08:59:21.735661+0000","last_active":"2024-01-20T08:59:21.735639+0000","last_peered":"2024-01-20T08:59:21.735639+0000","last_clean":"2024-01-20T08:59:21.735639+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:59:21.735639+0000","last_undegraded":"2024-01-20T08:59:21.735639+0000","last_fullsized":"2024-01-20T08:59:21.735639+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'162","last_scrub_stamp":"2024-01-20T08:59:21.735574+0000","last_deep_scrub":"27'162","last_deep_scrub_stamp":"2024-01-20T08:59:21.735574+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.735574+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:23:33.326267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":10773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'697","reported_seq":739,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.037427+0000","last_change":"2024-01-20T08:59:14.037427+0000","last_active":"2024-01-20T08:59:14.037427+0000","last_peered":"2024-01-20T08:59:14.037427+0000","last_clean":"2024-01-20T08:59:14.037427+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:59:14.037427+0000","last_undegraded":"2024-01-20T08:59:14.037427+0000","last_fullsized":"2024-01-20T08:59:14.037427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'697","last_scrub_stamp":"2024-01-20T08:59:14.037353+0000","last_deep_scrub":"27'697","last_deep_scrub_stamp":"2024-01-20T08:59:14.037353+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.037353+0000","objects_scrubbed":1,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:00:11.324134+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"27'37","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.991343+0000","last_change":"2024-01-20T08:59:19.991382+0000","last_active":"2024-01-20T08:59:19.991343+0000","last_peered":"2024-01-20T08:59:19.991343+0000","last_clean":"2024-01-20T08:59:19.991343+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:59:19.991343+0000","last_undegraded":"2024-01-20T08:59:19.991343+0000","last_fullsized":"2024-01-20T08:59:19.991343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:19.991267+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:19.991267+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.991267+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:36.266081+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"27'136","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.069252+0000","last_change":"2024-01-20T08:59:15.069277+0000","last_active":"2024-01-20T08:59:15.069252+0000","last_peered":"2024-01-20T08:59:15.069252+0000","last_clean":"2024-01-20T08:59:15.069252+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:59:15.069252+0000","last_undegraded":"2024-01-20T08:59:15.069252+0000","last_fullsized":"2024-01-20T08:59:15.069252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'136","last_scrub_stamp":"2024-01-20T08:59:15.069194+0000","last_deep_scrub":"27'136","last_deep_scrub_stamp":"2024-01-20T08:59:15.069194+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.069194+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:25.628098+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'1361","reported_seq":1400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.841575+0000","last_change":"2024-01-20T08:59:11.841614+0000","last_active":"2024-01-20T08:59:11.841575+0000","last_peered":"2024-01-20T08:59:11.841575+0000","last_clean":"2024-01-20T08:59:11.841575+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:59:11.841575+0000","last_undegraded":"2024-01-20T08:59:11.841575+0000","last_fullsized":"2024-01-20T08:59:11.841575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1361","last_scrub_stamp":"2024-01-20T08:59:11.841510+0000","last_deep_scrub":"27'1361","last_deep_scrub_stamp":"2024-01-20T08:59:11.841510+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.841510+0000","objects_scrubbed":9,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:29:55.568602+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"27'31","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.841361+0000","last_change":"2024-01-20T08:59:09.841382+0000","last_active":"2024-01-20T08:59:09.841361+0000","last_peered":"2024-01-20T08:59:09.841361+0000","last_clean":"2024-01-20T08:59:09.841361+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:59:09.841361+0000","last_undegraded":"2024-01-20T08:59:09.841361+0000","last_fullsized":"2024-01-20T08:59:09.841361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:09.841311+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:09.841311+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.841311+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:03:42.035846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"27'128","reported_seq":163,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.173162+0000","last_change":"2024-01-20T08:59:19.173186+0000","last_active":"2024-01-20T08:59:19.173162+0000","last_peered":"2024-01-20T08:59:19.173162+0000","last_clean":"2024-01-20T08:59:19.173162+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:59:19.173162+0000","last_undegraded":"2024-01-20T08:59:19.173162+0000","last_fullsized":"2024-01-20T08:59:19.173162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'128","last_scrub_stamp":"2024-01-20T08:59:19.173096+0000","last_deep_scrub":"27'128","last_deep_scrub_stamp":"2024-01-20T08:59:19.173096+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.173096+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:02:49.049464+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'11","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.770188+0000","last_change":"2024-01-20T08:59:09.770234+0000","last_active":"2024-01-20T08:59:09.770188+0000","last_peered":"2024-01-20T08:59:09.770188+0000","last_clean":"2024-01-20T08:59:09.770188+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:59:09.770188+0000","last_undegraded":"2024-01-20T08:59:09.770188+0000","last_fullsized":"2024-01-20T08:59:09.770188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-01-20T08:59:09.770124+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-01-20T08:59:09.770124+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.770124+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:39.811326+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"27'47","reported_seq":85,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.475087+0000","last_change":"2024-01-20T08:59:08.475109+0000","last_active":"2024-01-20T08:59:08.475087+0000","last_peered":"2024-01-20T08:59:08.475087+0000","last_clean":"2024-01-20T08:59:08.475087+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:59:08.475087+0000","last_undegraded":"2024-01-20T08:59:08.475087+0000","last_fullsized":"2024-01-20T08:59:08.475087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-01-20T08:59:08.475024+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-01-20T08:59:08.475024+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.475024+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:26:07.581884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"27'149","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.144297+0000","last_change":"2024-01-20T08:59:18.144342+0000","last_active":"2024-01-20T08:59:18.144297+0000","last_peered":"2024-01-20T08:59:18.144297+0000","last_clean":"2024-01-20T08:59:18.144297+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:59:18.144297+0000","last_undegraded":"2024-01-20T08:59:18.144297+0000","last_fullsized":"2024-01-20T08:59:18.144297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'149","last_scrub_stamp":"2024-01-20T08:59:18.144206+0000","last_deep_scrub":"27'149","last_deep_scrub_stamp":"2024-01-20T08:59:18.144206+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.144206+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:49.578407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":7763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'35","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:25.759239+0000","last_change":"2024-01-20T08:59:25.759258+0000","last_active":"2024-01-20T08:59:25.759239+0000","last_peered":"2024-01-20T08:59:25.759239+0000","last_clean":"2024-01-20T08:59:25.759239+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:59:25.759239+0000","last_undegraded":"2024-01-20T08:59:25.759239+0000","last_fullsized":"2024-01-20T08:59:25.759239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:25.759177+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:25.759177+0000","last_clean_scrub_stamp":"2024-01-20T08:59:25.759177+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:26:20.633472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'596","reported_seq":636,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.539017+0000","last_change":"2024-01-20T08:59:15.539043+0000","last_active":"2024-01-20T08:59:15.539017+0000","last_peered":"2024-01-20T08:59:15.539017+0000","last_clean":"2024-01-20T08:59:15.539017+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:59:15.539017+0000","last_undegraded":"2024-01-20T08:59:15.539017+0000","last_fullsized":"2024-01-20T08:59:15.539017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'596","last_scrub_stamp":"2024-01-20T08:59:15.538958+0000","last_deep_scrub":"27'596","last_deep_scrub_stamp":"2024-01-20T08:59:15.538958+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.538958+0000","objects_scrubbed":3,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:45:35.288982+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":4387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'109","reported_seq":145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:28.847038+0000","last_change":"2024-01-20T08:59:28.847078+0000","last_active":"2024-01-20T08:59:28.847038+0000","last_peered":"2024-01-20T08:59:28.847038+0000","last_clean":"2024-01-20T08:59:28.847038+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:59:28.847038+0000","last_undegraded":"2024-01-20T08:59:28.847038+0000","last_fullsized":"2024-01-20T08:59:28.847038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'109","last_scrub_stamp":"2024-01-20T08:59:28.846943+0000","last_deep_scrub":"27'109","last_deep_scrub_stamp":"2024-01-20T08:59:28.846943+0000","last_clean_scrub_stamp":"2024-01-20T08:59:28.846943+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:39:37.780732+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"27'35","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.216375+0000","last_change":"2024-01-20T08:59:15.216397+0000","last_active":"2024-01-20T08:59:15.216375+0000","last_peered":"2024-01-20T08:59:15.216375+0000","last_clean":"2024-01-20T08:59:15.216375+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:59:15.216375+0000","last_undegraded":"2024-01-20T08:59:15.216375+0000","last_fullsized":"2024-01-20T08:59:15.216375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:15.216304+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:15.216304+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.216304+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:38:51.318528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'8","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.372586+0000","last_change":"2024-01-20T08:59:11.372586+0000","last_active":"2024-01-20T08:59:11.372586+0000","last_peered":"2024-01-20T08:59:11.372586+0000","last_clean":"2024-01-20T08:59:11.372586+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:59:11.372586+0000","last_undegraded":"2024-01-20T08:59:11.372586+0000","last_fullsized":"2024-01-20T08:59:11.372586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:11.372441+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:11.372441+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.372441+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:58:44.179767+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"27'149","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.728023+0000","last_change":"2024-01-20T08:59:20.728046+0000","last_active":"2024-01-20T08:59:20.728023+0000","last_peered":"2024-01-20T08:59:20.728023+0000","last_clean":"2024-01-20T08:59:20.728023+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:59:20.728023+0000","last_undegraded":"2024-01-20T08:59:20.728023+0000","last_fullsized":"2024-01-20T08:59:20.728023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'149","last_scrub_stamp":"2024-01-20T08:59:20.727963+0000","last_deep_scrub":"27'149","last_deep_scrub_stamp":"2024-01-20T08:59:20.727963+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.727963+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:18:29.451689+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'34","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.697472+0000","last_change":"2024-01-20T08:59:07.697472+0000","last_active":"2024-01-20T08:59:07.697472+0000","last_peered":"2024-01-20T08:59:07.697472+0000","last_clean":"2024-01-20T08:59:07.697472+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:59:07.697472+0000","last_undegraded":"2024-01-20T08:59:07.697472+0000","last_fullsized":"2024-01-20T08:59:07.697472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:07.697392+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:07.697392+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.697392+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:46:28.055701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'17","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.680417+0000","last_change":"2024-01-20T08:59:09.680417+0000","last_active":"2024-01-20T08:59:09.680417+0000","last_peered":"2024-01-20T08:59:09.680417+0000","last_clean":"2024-01-20T08:59:09.680417+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:59:09.680417+0000","last_undegraded":"2024-01-20T08:59:09.680417+0000","last_fullsized":"2024-01-20T08:59:09.680417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-01-20T08:59:09.680291+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-01-20T08:59:09.680291+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.680291+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:40:27.031404+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"27'121","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.195601+0000","last_change":"2024-01-20T08:59:16.195626+0000","last_active":"2024-01-20T08:59:16.195601+0000","last_peered":"2024-01-20T08:59:16.195601+0000","last_clean":"2024-01-20T08:59:16.195601+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:59:16.195601+0000","last_undegraded":"2024-01-20T08:59:16.195601+0000","last_fullsized":"2024-01-20T08:59:16.195601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'121","last_scrub_stamp":"2024-01-20T08:59:16.195536+0000","last_deep_scrub":"27'121","last_deep_scrub_stamp":"2024-01-20T08:59:16.195536+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.195536+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:24:51.756954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'32","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.526131+0000","last_change":"2024-01-20T08:59:12.526131+0000","last_active":"2024-01-20T08:59:12.526131+0000","last_peered":"2024-01-20T08:59:12.526131+0000","last_clean":"2024-01-20T08:59:12.526131+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:59:12.526131+0000","last_undegraded":"2024-01-20T08:59:12.526131+0000","last_fullsized":"2024-01-20T08:59:12.526131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'32","last_scrub_stamp":"2024-01-20T08:59:12.526042+0000","last_deep_scrub":"27'32","last_deep_scrub_stamp":"2024-01-20T08:59:12.526042+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.526042+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:07:13.378040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"27'8","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.785080+0000","last_change":"2024-01-20T08:59:08.785080+0000","last_active":"2024-01-20T08:59:08.785080+0000","last_peered":"2024-01-20T08:59:08.785080+0000","last_clean":"2024-01-20T08:59:08.785080+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:59:08.785080+0000","last_undegraded":"2024-01-20T08:59:08.785080+0000","last_fullsized":"2024-01-20T08:59:08.785080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:08.784998+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:08.784998+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.784998+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:51.057586+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'112","reported_seq":148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.191749+0000","last_change":"2024-01-20T08:59:20.191788+0000","last_active":"2024-01-20T08:59:20.191749+0000","last_peered":"2024-01-20T08:59:20.191749+0000","last_clean":"2024-01-20T08:59:20.191749+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:59:20.191749+0000","last_undegraded":"2024-01-20T08:59:20.191749+0000","last_fullsized":"2024-01-20T08:59:20.191749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'112","last_scrub_stamp":"2024-01-20T08:59:20.191680+0000","last_deep_scrub":"27'112","last_deep_scrub_stamp":"2024-01-20T08:59:20.191680+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.191680+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:43:20.989203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'41","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.734360+0000","last_change":"2024-01-20T08:59:16.734360+0000","last_active":"2024-01-20T08:59:16.734360+0000","last_peered":"2024-01-20T08:59:16.734360+0000","last_clean":"2024-01-20T08:59:16.734360+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:59:16.734360+0000","last_undegraded":"2024-01-20T08:59:16.734360+0000","last_fullsized":"2024-01-20T08:59:16.734360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:16.734239+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:16.734239+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.734239+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:04:37.910402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.601709+0000","last_change":"2024-01-20T08:59:18.601709+0000","last_active":"2024-01-20T08:59:18.601709+0000","last_peered":"2024-01-20T08:59:18.601709+0000","last_clean":"2024-01-20T08:59:18.601709+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:59:18.601709+0000","last_undegraded":"2024-01-20T08:59:18.601709+0000","last_fullsized":"2024-01-20T08:59:18.601709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-01-20T08:59:18.601614+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-01-20T08:59:18.601614+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.601614+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:32:05.733388+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"27'148","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.122528+0000","last_change":"2024-01-20T08:59:17.122572+0000","last_active":"2024-01-20T08:59:17.122528+0000","last_peered":"2024-01-20T08:59:17.122528+0000","last_clean":"2024-01-20T08:59:17.122528+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:59:17.122528+0000","last_undegraded":"2024-01-20T08:59:17.122528+0000","last_fullsized":"2024-01-20T08:59:17.122528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'148","last_scrub_stamp":"2024-01-20T08:59:17.122452+0000","last_deep_scrub":"27'148","last_deep_scrub_stamp":"2024-01-20T08:59:17.122452+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.122452+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:55:19.000693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'36","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.699753+0000","last_change":"2024-01-20T08:59:18.699753+0000","last_active":"2024-01-20T08:59:18.699753+0000","last_peered":"2024-01-20T08:59:18.699753+0000","last_clean":"2024-01-20T08:59:18.699753+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:59:18.699753+0000","last_undegraded":"2024-01-20T08:59:18.699753+0000","last_fullsized":"2024-01-20T08:59:18.699753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:18.699669+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:18.699669+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.699669+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:59:54.426206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'318","reported_seq":359,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.680992+0000","last_change":"2024-01-20T08:59:10.680992+0000","last_active":"2024-01-20T08:59:10.680992+0000","last_peered":"2024-01-20T08:59:10.680992+0000","last_clean":"2024-01-20T08:59:10.680992+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:59:10.680992+0000","last_undegraded":"2024-01-20T08:59:10.680992+0000","last_fullsized":"2024-01-20T08:59:10.680992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'318","last_scrub_stamp":"2024-01-20T08:59:10.680900+0000","last_deep_scrub":"27'318","last_deep_scrub_stamp":"2024-01-20T08:59:10.680900+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.680900+0000","objects_scrubbed":4,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:02:01.093433+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"27'148","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.715552+0000","last_change":"2024-01-20T08:59:23.715574+0000","last_active":"2024-01-20T08:59:23.715552+0000","last_peered":"2024-01-20T08:59:23.715552+0000","last_clean":"2024-01-20T08:59:23.715552+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:59:23.715552+0000","last_undegraded":"2024-01-20T08:59:23.715552+0000","last_fullsized":"2024-01-20T08:59:23.715552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'148","last_scrub_stamp":"2024-01-20T08:59:23.715478+0000","last_deep_scrub":"27'148","last_deep_scrub_stamp":"2024-01-20T08:59:23.715478+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.715478+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:03:45.517044+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":11573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"27'35","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.353399+0000","last_change":"2024-01-20T08:59:15.353441+0000","last_active":"2024-01-20T08:59:15.353399+0000","last_peered":"2024-01-20T08:59:15.353399+0000","last_clean":"2024-01-20T08:59:15.353399+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:59:15.353399+0000","last_undegraded":"2024-01-20T08:59:15.353399+0000","last_fullsized":"2024-01-20T08:59:15.353399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:15.353322+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:15.353322+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.353322+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:45.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"27'11","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.627602+0000","last_change":"2024-01-20T08:59:20.627628+0000","last_active":"2024-01-20T08:59:20.627602+0000","last_peered":"2024-01-20T08:59:20.627602+0000","last_clean":"2024-01-20T08:59:20.627602+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:59:20.627602+0000","last_undegraded":"2024-01-20T08:59:20.627602+0000","last_fullsized":"2024-01-20T08:59:20.627602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-01-20T08:59:20.627551+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-01-20T08:59:20.627551+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.627551+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:30:21.993177+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"27'126","reported_seq":163,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.293095+0000","last_change":"2024-01-20T08:59:22.293118+0000","last_active":"2024-01-20T08:59:22.293095+0000","last_peered":"2024-01-20T08:59:22.293095+0000","last_clean":"2024-01-20T08:59:22.293095+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:59:22.293095+0000","last_undegraded":"2024-01-20T08:59:22.293095+0000","last_fullsized":"2024-01-20T08:59:22.293095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'126","last_scrub_stamp":"2024-01-20T08:59:22.293023+0000","last_deep_scrub":"27'126","last_deep_scrub_stamp":"2024-01-20T08:59:22.293023+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.293023+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:52.565087+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'33","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.069746+0000","last_change":"2024-01-20T08:59:13.069774+0000","last_active":"2024-01-20T08:59:13.069746+0000","last_peered":"2024-01-20T08:59:13.069746+0000","last_clean":"2024-01-20T08:59:13.069746+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:59:13.069746+0000","last_undegraded":"2024-01-20T08:59:13.069746+0000","last_fullsized":"2024-01-20T08:59:13.069746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:13.069682+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:13.069682+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.069682+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:55.342270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.986888+0000","last_change":"2024-01-20T08:59:11.986951+0000","last_active":"2024-01-20T08:59:11.986888+0000","last_peered":"2024-01-20T08:59:11.986888+0000","last_clean":"2024-01-20T08:59:11.986888+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:59:11.986888+0000","last_undegraded":"2024-01-20T08:59:11.986888+0000","last_fullsized":"2024-01-20T08:59:11.986888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-01-20T08:59:11.986774+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-01-20T08:59:11.986774+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.986774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:42.027975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"27'181","reported_seq":218,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.342183+0000","last_change":"2024-01-20T08:59:21.342206+0000","last_active":"2024-01-20T08:59:21.342183+0000","last_peered":"2024-01-20T08:59:21.342183+0000","last_clean":"2024-01-20T08:59:21.342183+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:59:21.342183+0000","last_undegraded":"2024-01-20T08:59:21.342183+0000","last_fullsized":"2024-01-20T08:59:21.342183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'181","last_scrub_stamp":"2024-01-20T08:59:21.342122+0000","last_deep_scrub":"27'181","last_deep_scrub_stamp":"2024-01-20T08:59:21.342122+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.342122+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:33:15.086065+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"27'33","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.773565+0000","last_change":"2024-01-20T08:59:13.773603+0000","last_active":"2024-01-20T08:59:13.773565+0000","last_peered":"2024-01-20T08:59:13.773565+0000","last_clean":"2024-01-20T08:59:13.773565+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:59:13.773565+0000","last_undegraded":"2024-01-20T08:59:13.773565+0000","last_fullsized":"2024-01-20T08:59:13.773565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:13.773482+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:13.773482+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.773482+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:38:42.376489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"27'15","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.624391+0000","last_change":"2024-01-20T08:59:19.624413+0000","last_active":"2024-01-20T08:59:19.624391+0000","last_peered":"2024-01-20T08:59:19.624391+0000","last_clean":"2024-01-20T08:59:19.624391+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:59:19.624391+0000","last_undegraded":"2024-01-20T08:59:19.624391+0000","last_fullsized":"2024-01-20T08:59:19.624391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-01-20T08:59:19.624338+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-01-20T08:59:19.624338+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.624338+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:26:26.722625+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'113","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.305447+0000","last_change":"2024-01-20T08:59:20.305469+0000","last_active":"2024-01-20T08:59:20.305447+0000","last_peered":"2024-01-20T08:59:20.305447+0000","last_clean":"2024-01-20T08:59:20.305447+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:59:20.305447+0000","last_undegraded":"2024-01-20T08:59:20.305447+0000","last_fullsized":"2024-01-20T08:59:20.305447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'113","last_scrub_stamp":"2024-01-20T08:59:20.305377+0000","last_deep_scrub":"27'113","last_deep_scrub_stamp":"2024-01-20T08:59:20.305377+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.305377+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:59:56.031913+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"27'32","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.074739+0000","last_change":"2024-01-20T08:59:08.074739+0000","last_active":"2024-01-20T08:59:08.074739+0000","last_peered":"2024-01-20T08:59:08.074739+0000","last_clean":"2024-01-20T08:59:08.074739+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:59:08.074739+0000","last_undegraded":"2024-01-20T08:59:08.074739+0000","last_fullsized":"2024-01-20T08:59:08.074739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'32","last_scrub_stamp":"2024-01-20T08:59:08.074572+0000","last_deep_scrub":"27'32","last_deep_scrub_stamp":"2024-01-20T08:59:08.074572+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.074572+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:37:10.110970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"27'980","reported_seq":1988,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.303347+0000","last_change":"2024-01-20T08:59:12.303372+0000","last_active":"2024-01-20T08:59:12.303347+0000","last_peered":"2024-01-20T08:59:12.303347+0000","last_clean":"2024-01-20T08:59:12.303347+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:59:12.303347+0000","last_undegraded":"2024-01-20T08:59:12.303347+0000","last_fullsized":"2024-01-20T08:59:12.303347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'980","last_scrub_stamp":"2024-01-20T08:59:12.303280+0000","last_deep_scrub":"27'980","last_deep_scrub_stamp":"2024-01-20T08:59:12.303280+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.303280+0000","objects_scrubbed":4,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:43:14.836075+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210232,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":976,"num_write_kb":1034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"27'135","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.341955+0000","last_change":"2024-01-20T08:59:23.341955+0000","last_active":"2024-01-20T08:59:23.341955+0000","last_peered":"2024-01-20T08:59:23.341955+0000","last_clean":"2024-01-20T08:59:23.341955+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:59:23.341955+0000","last_undegraded":"2024-01-20T08:59:23.341955+0000","last_fullsized":"2024-01-20T08:59:23.341955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'135","last_scrub_stamp":"2024-01-20T08:59:23.341833+0000","last_deep_scrub":"27'135","last_deep_scrub_stamp":"2024-01-20T08:59:23.341833+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.341833+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:39.616646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118,"num_write_kb":10261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"27'28","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.373546+0000","last_change":"2024-01-20T08:59:17.373581+0000","last_active":"2024-01-20T08:59:17.373546+0000","last_peered":"2024-01-20T08:59:17.373546+0000","last_clean":"2024-01-20T08:59:17.373546+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:59:17.373546+0000","last_undegraded":"2024-01-20T08:59:17.373546+0000","last_fullsized":"2024-01-20T08:59:17.373546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-01-20T08:59:17.373441+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-01-20T08:59:17.373441+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.373441+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:30:01.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"27'431","reported_seq":473,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.668712+0000","last_change":"2024-01-20T08:59:21.668735+0000","last_active":"2024-01-20T08:59:21.668712+0000","last_peered":"2024-01-20T08:59:21.668712+0000","last_clean":"2024-01-20T08:59:21.668712+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:59:21.668712+0000","last_undegraded":"2024-01-20T08:59:21.668712+0000","last_fullsized":"2024-01-20T08:59:21.668712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'431","last_scrub_stamp":"2024-01-20T08:59:21.668659+0000","last_deep_scrub":"27'431","last_deep_scrub_stamp":"2024-01-20T08:59:21.668659+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.668659+0000","objects_scrubbed":2,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:57:22.141144+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":4309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"27'129","reported_seq":166,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:24.051094+0000","last_change":"2024-01-20T08:59:24.051094+0000","last_active":"2024-01-20T08:59:24.051094+0000","last_peered":"2024-01-20T08:59:24.051094+0000","last_clean":"2024-01-20T08:59:24.051094+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:59:24.051094+0000","last_undegraded":"2024-01-20T08:59:24.051094+0000","last_fullsized":"2024-01-20T08:59:24.051094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'129","last_scrub_stamp":"2024-01-20T08:59:24.050992+0000","last_deep_scrub":"27'129","last_deep_scrub_stamp":"2024-01-20T08:59:24.050992+0000","last_clean_scrub_stamp":"2024-01-20T08:59:24.050992+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:10.540045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'32","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.693053+0000","last_change":"2024-01-20T08:59:14.693089+0000","last_active":"2024-01-20T08:59:14.693053+0000","last_peered":"2024-01-20T08:59:14.693053+0000","last_clean":"2024-01-20T08:59:14.693053+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:59:14.693053+0000","last_undegraded":"2024-01-20T08:59:14.693053+0000","last_fullsized":"2024-01-20T08:59:14.693053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'32","last_scrub_stamp":"2024-01-20T08:59:14.692967+0000","last_deep_scrub":"27'32","last_deep_scrub_stamp":"2024-01-20T08:59:14.692967+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.692967+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:13:16.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.675957+0000","last_change":"2024-01-20T08:59:23.676000+0000","last_active":"2024-01-20T08:59:23.675957+0000","last_peered":"2024-01-20T08:59:23.675957+0000","last_clean":"2024-01-20T08:59:23.675957+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:59:23.675957+0000","last_undegraded":"2024-01-20T08:59:23.675957+0000","last_fullsized":"2024-01-20T08:59:23.675957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-01-20T08:59:23.675897+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-01-20T08:59:23.675897+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.675897+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:42:54.610826+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":197,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:05.824186+0000","last_change":"2024-01-20T08:59:05.824213+0000","last_active":"2024-01-20T08:59:05.824186+0000","last_peered":"2024-01-20T08:59:05.824186+0000","last_clean":"2024-01-20T08:59:05.824186+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:59:05.824186+0000","last_undegraded":"2024-01-20T08:59:05.824186+0000","last_fullsized":"2024-01-20T08:59:05.824186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-01-20T08:59:05.824134+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-01-20T08:59:05.824134+0000","last_clean_scrub_stamp":"2024-01-20T08:59:05.824134+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:29:17.877275+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'156","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.867902+0000","last_change":"2024-01-20T08:31:16.176252+0000","last_active":"2024-01-20T08:58:25.867902+0000","last_peered":"2024-01-20T08:58:25.867902+0000","last_clean":"2024-01-20T08:58:25.867902+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:58:25.867902+0000","last_undegraded":"2024-01-20T08:58:25.867902+0000","last_fullsized":"2024-01-20T08:58:25.867902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":9711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"27'36","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:26.759230+0000","last_change":"2024-01-20T08:59:26.759275+0000","last_active":"2024-01-20T08:59:26.759230+0000","last_peered":"2024-01-20T08:59:26.759230+0000","last_clean":"2024-01-20T08:59:26.759230+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:59:26.759230+0000","last_undegraded":"2024-01-20T08:59:26.759230+0000","last_fullsized":"2024-01-20T08:59:26.759230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:26.759165+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:26.759165+0000","last_clean_scrub_stamp":"2024-01-20T08:59:26.759165+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:07:43.517630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"27'10","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.374340+0000","last_change":"2024-01-20T08:59:09.374364+0000","last_active":"2024-01-20T08:59:09.374340+0000","last_peered":"2024-01-20T08:59:09.374340+0000","last_clean":"2024-01-20T08:59:09.374340+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:59:09.374340+0000","last_undegraded":"2024-01-20T08:59:09.374340+0000","last_fullsized":"2024-01-20T08:59:09.374340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-01-20T08:59:09.374281+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-01-20T08:59:09.374281+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.374281+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:23:20.175940+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"27'130","reported_seq":166,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.098283+0000","last_change":"2024-01-20T08:59:16.098305+0000","last_active":"2024-01-20T08:59:16.098283+0000","last_peered":"2024-01-20T08:59:16.098283+0000","last_clean":"2024-01-20T08:59:16.098283+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:59:16.098283+0000","last_undegraded":"2024-01-20T08:59:16.098283+0000","last_fullsized":"2024-01-20T08:59:16.098283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-01-20T08:59:16.098224+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-01-20T08:59:16.098224+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.098224+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:00:13.385005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":4819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'39","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.775637+0000","last_change":"2024-01-20T08:59:15.775659+0000","last_active":"2024-01-20T08:59:15.775637+0000","last_peered":"2024-01-20T08:59:15.775637+0000","last_clean":"2024-01-20T08:59:15.775637+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:59:15.775637+0000","last_undegraded":"2024-01-20T08:59:15.775637+0000","last_fullsized":"2024-01-20T08:59:15.775637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:15.775578+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:15.775578+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.775578+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:33:09.519522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'7","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.046779+0000","last_change":"2024-01-20T08:59:15.046805+0000","last_active":"2024-01-20T08:59:15.046779+0000","last_peered":"2024-01-20T08:59:15.046779+0000","last_clean":"2024-01-20T08:59:15.046779+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:59:15.046779+0000","last_undegraded":"2024-01-20T08:59:15.046779+0000","last_fullsized":"2024-01-20T08:59:15.046779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-01-20T08:59:15.046715+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-01-20T08:59:15.046715+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.046715+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:55:16.924956+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"27'188","reported_seq":224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:26.093943+0000","last_change":"2024-01-20T08:59:26.093971+0000","last_active":"2024-01-20T08:59:26.093943+0000","last_peered":"2024-01-20T08:59:26.093943+0000","last_clean":"2024-01-20T08:59:26.093943+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:59:26.093943+0000","last_undegraded":"2024-01-20T08:59:26.093943+0000","last_fullsized":"2024-01-20T08:59:26.093943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'188","last_scrub_stamp":"2024-01-20T08:59:26.093851+0000","last_deep_scrub":"27'188","last_deep_scrub_stamp":"2024-01-20T08:59:26.093851+0000","last_clean_scrub_stamp":"2024-01-20T08:59:26.093851+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:38:36.846978+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.150876+0000","last_change":"2024-01-20T08:59:19.150932+0000","last_active":"2024-01-20T08:59:19.150876+0000","last_peered":"2024-01-20T08:59:19.150876+0000","last_clean":"2024-01-20T08:59:19.150876+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:59:19.150876+0000","last_undegraded":"2024-01-20T08:59:19.150876+0000","last_fullsized":"2024-01-20T08:59:19.150876+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:19.150805+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:19.150805+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.150805+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:39:22.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'10","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.722710+0000","last_change":"2024-01-20T08:59:12.722733+0000","last_active":"2024-01-20T08:59:12.722710+0000","last_peered":"2024-01-20T08:59:12.722710+0000","last_clean":"2024-01-20T08:59:12.722710+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:59:12.722710+0000","last_undegraded":"2024-01-20T08:59:12.722710+0000","last_fullsized":"2024-01-20T08:59:12.722710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-01-20T08:59:12.722653+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-01-20T08:59:12.722653+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.722653+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:54.995241+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"27'145","reported_seq":174,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.438853+0000","last_change":"2024-01-20T08:31:16.179852+0000","last_active":"2024-01-20T08:59:06.438853+0000","last_peered":"2024-01-20T08:59:06.438853+0000","last_clean":"2024-01-20T08:59:06.438853+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:59:06.438853+0000","last_undegraded":"2024-01-20T08:59:06.438853+0000","last_fullsized":"2024-01-20T08:59:06.438853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'21","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.049300+0000","last_change":"2024-01-20T08:59:09.049341+0000","last_active":"2024-01-20T08:59:09.049300+0000","last_peered":"2024-01-20T08:59:09.049300+0000","last_clean":"2024-01-20T08:59:09.049300+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:59:09.049300+0000","last_undegraded":"2024-01-20T08:59:09.049300+0000","last_fullsized":"2024-01-20T08:59:09.049300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-01-20T08:59:09.049204+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-01-20T08:59:09.049204+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.049204+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:19.502084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'5","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.740446+0000","last_change":"2024-01-20T08:59:07.740470+0000","last_active":"2024-01-20T08:59:07.740446+0000","last_peered":"2024-01-20T08:59:07.740446+0000","last_clean":"2024-01-20T08:59:07.740446+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:59:07.740446+0000","last_undegraded":"2024-01-20T08:59:07.740446+0000","last_fullsized":"2024-01-20T08:59:07.740446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-01-20T08:59:07.740394+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-01-20T08:59:07.740394+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.740394+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:21.774310+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"27'160","reported_seq":197,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.877502+0000","last_change":"2024-01-20T08:59:12.877502+0000","last_active":"2024-01-20T08:59:12.877502+0000","last_peered":"2024-01-20T08:59:12.877502+0000","last_clean":"2024-01-20T08:59:12.877502+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:59:12.877502+0000","last_undegraded":"2024-01-20T08:59:12.877502+0000","last_fullsized":"2024-01-20T08:59:12.877502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'160","last_scrub_stamp":"2024-01-20T08:59:12.877413+0000","last_deep_scrub":"27'160","last_deep_scrub_stamp":"2024-01-20T08:59:12.877413+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.877413+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:23.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":10942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"27'35","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.547738+0000","last_change":"2024-01-20T08:59:13.547770+0000","last_active":"2024-01-20T08:59:13.547738+0000","last_peered":"2024-01-20T08:59:13.547738+0000","last_clean":"2024-01-20T08:59:13.547738+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:59:13.547738+0000","last_undegraded":"2024-01-20T08:59:13.547738+0000","last_fullsized":"2024-01-20T08:59:13.547738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:13.547656+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:13.547656+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.547656+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:55:30.322108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'29","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.679673+0000","last_change":"2024-01-20T08:59:08.679673+0000","last_active":"2024-01-20T08:59:08.679673+0000","last_peered":"2024-01-20T08:59:08.679673+0000","last_clean":"2024-01-20T08:59:08.679673+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:59:08.679673+0000","last_undegraded":"2024-01-20T08:59:08.679673+0000","last_fullsized":"2024-01-20T08:59:08.679673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-01-20T08:59:08.679555+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-01-20T08:59:08.679555+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.679555+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:58:41.486881+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"27'142","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:25.047670+0000","last_change":"2024-01-20T08:59:25.047717+0000","last_active":"2024-01-20T08:59:25.047670+0000","last_peered":"2024-01-20T08:59:25.047670+0000","last_clean":"2024-01-20T08:59:25.047670+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:59:25.047670+0000","last_undegraded":"2024-01-20T08:59:25.047670+0000","last_fullsized":"2024-01-20T08:59:25.047670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'142","last_scrub_stamp":"2024-01-20T08:59:25.047568+0000","last_deep_scrub":"27'142","last_deep_scrub_stamp":"2024-01-20T08:59:25.047568+0000","last_clean_scrub_stamp":"2024-01-20T08:59:25.047568+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:35:06.679443+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'29","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.785242+0000","last_change":"2024-01-20T08:59:07.785242+0000","last_active":"2024-01-20T08:59:07.785242+0000","last_peered":"2024-01-20T08:59:07.785242+0000","last_clean":"2024-01-20T08:59:07.785242+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:59:07.785242+0000","last_undegraded":"2024-01-20T08:59:07.785242+0000","last_fullsized":"2024-01-20T08:59:07.785242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-01-20T08:59:07.785115+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-01-20T08:59:07.785115+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.785115+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:03:31.674517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"27'13","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.790415+0000","last_change":"2024-01-20T08:59:06.790415+0000","last_active":"2024-01-20T08:59:06.790415+0000","last_peered":"2024-01-20T08:59:06.790415+0000","last_clean":"2024-01-20T08:59:06.790415+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:59:06.790415+0000","last_undegraded":"2024-01-20T08:59:06.790415+0000","last_fullsized":"2024-01-20T08:59:06.790415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-01-20T08:59:06.790291+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-01-20T08:59:06.790291+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.790291+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:57.202825+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1169,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"27'133","reported_seq":160,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:23.559999+0000","last_change":"2024-01-20T08:31:16.178761+0000","last_active":"2024-01-20T08:58:23.559999+0000","last_peered":"2024-01-20T08:58:23.559999+0000","last_clean":"2024-01-20T08:58:23.559999+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:58:23.559999+0000","last_undegraded":"2024-01-20T08:58:23.559999+0000","last_fullsized":"2024-01-20T08:58:23.559999+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'48","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.483859+0000","last_change":"2024-01-20T08:59:07.483859+0000","last_active":"2024-01-20T08:59:07.483859+0000","last_peered":"2024-01-20T08:59:07.483859+0000","last_clean":"2024-01-20T08:59:07.483859+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:59:07.483859+0000","last_undegraded":"2024-01-20T08:59:07.483859+0000","last_fullsized":"2024-01-20T08:59:07.483859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-01-20T08:59:07.483752+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-01-20T08:59:07.483752+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.483752+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:55:39.655254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.570873+0000","last_change":"2024-01-20T08:59:16.570914+0000","last_active":"2024-01-20T08:59:16.570873+0000","last_peered":"2024-01-20T08:59:16.570873+0000","last_clean":"2024-01-20T08:59:16.570873+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:59:16.570873+0000","last_undegraded":"2024-01-20T08:59:16.570873+0000","last_fullsized":"2024-01-20T08:59:16.570873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-01-20T08:59:16.570787+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-01-20T08:59:16.570787+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.570787+0000","objects_scrubbed":3,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:48:17.320248+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"27'116","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.330579+0000","last_change":"2024-01-20T08:59:19.330579+0000","last_active":"2024-01-20T08:59:19.330579+0000","last_peered":"2024-01-20T08:59:19.330579+0000","last_clean":"2024-01-20T08:59:19.330579+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:59:19.330579+0000","last_undegraded":"2024-01-20T08:59:19.330579+0000","last_fullsized":"2024-01-20T08:59:19.330579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'116","last_scrub_stamp":"2024-01-20T08:59:19.330498+0000","last_deep_scrub":"27'116","last_deep_scrub_stamp":"2024-01-20T08:59:19.330498+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.330498+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:38:47.933506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"27'40","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.393935+0000","last_change":"2024-01-20T08:59:13.393974+0000","last_active":"2024-01-20T08:59:13.393935+0000","last_peered":"2024-01-20T08:59:13.393935+0000","last_clean":"2024-01-20T08:59:13.393935+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:59:13.393935+0000","last_undegraded":"2024-01-20T08:59:13.393935+0000","last_fullsized":"2024-01-20T08:59:13.393935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'40","last_scrub_stamp":"2024-01-20T08:59:13.393836+0000","last_deep_scrub":"27'40","last_deep_scrub_stamp":"2024-01-20T08:59:13.393836+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.393836+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:27.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"27'117","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.293415+0000","last_change":"2024-01-20T08:59:10.293444+0000","last_active":"2024-01-20T08:59:10.293415+0000","last_peered":"2024-01-20T08:59:10.293415+0000","last_clean":"2024-01-20T08:59:10.293415+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:59:10.293415+0000","last_undegraded":"2024-01-20T08:59:10.293415+0000","last_fullsized":"2024-01-20T08:59:10.293415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'117","last_scrub_stamp":"2024-01-20T08:59:10.293359+0000","last_deep_scrub":"27'117","last_deep_scrub_stamp":"2024-01-20T08:59:10.293359+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.293359+0000","objects_scrubbed":7,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:29:41.075136+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069576,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"27'126","reported_seq":161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:27.803610+0000","last_change":"2024-01-20T08:59:27.803632+0000","last_active":"2024-01-20T08:59:27.803610+0000","last_peered":"2024-01-20T08:59:27.803610+0000","last_clean":"2024-01-20T08:59:27.803610+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:59:27.803610+0000","last_undegraded":"2024-01-20T08:59:27.803610+0000","last_fullsized":"2024-01-20T08:59:27.803610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'126","last_scrub_stamp":"2024-01-20T08:59:27.803543+0000","last_deep_scrub":"27'126","last_deep_scrub_stamp":"2024-01-20T08:59:27.803543+0000","last_clean_scrub_stamp":"2024-01-20T08:59:27.803543+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:59.209609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'36","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.713490+0000","last_change":"2024-01-20T08:59:19.713510+0000","last_active":"2024-01-20T08:59:19.713490+0000","last_peered":"2024-01-20T08:59:19.713490+0000","last_clean":"2024-01-20T08:59:19.713490+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:59:19.713490+0000","last_undegraded":"2024-01-20T08:59:19.713490+0000","last_fullsized":"2024-01-20T08:59:19.713490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:19.713435+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:19.713435+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.713435+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:26:17.789473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'838","reported_seq":878,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.086932+0000","last_change":"2024-01-20T08:59:06.086932+0000","last_active":"2024-01-20T08:59:06.086932+0000","last_peered":"2024-01-20T08:59:06.086932+0000","last_clean":"2024-01-20T08:59:06.086932+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:59:06.086932+0000","last_undegraded":"2024-01-20T08:59:06.086932+0000","last_fullsized":"2024-01-20T08:59:06.086932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'838","last_scrub_stamp":"2024-01-20T08:59:06.086846+0000","last_deep_scrub":"27'838","last_deep_scrub_stamp":"2024-01-20T08:59:06.086846+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.086846+0000","objects_scrubbed":6,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:03:38.281381+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"27'135","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.740285+0000","last_change":"2024-01-20T08:59:20.740325+0000","last_active":"2024-01-20T08:59:20.740285+0000","last_peered":"2024-01-20T08:59:20.740285+0000","last_clean":"2024-01-20T08:59:20.740285+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:59:20.740285+0000","last_undegraded":"2024-01-20T08:59:20.740285+0000","last_fullsized":"2024-01-20T08:59:20.740285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'135","last_scrub_stamp":"2024-01-20T08:59:20.740199+0000","last_deep_scrub":"27'135","last_deep_scrub_stamp":"2024-01-20T08:59:20.740199+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.740199+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:51.012263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"27'31","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.439085+0000","last_change":"2024-01-20T08:59:09.439085+0000","last_active":"2024-01-20T08:59:09.439085+0000","last_peered":"2024-01-20T08:59:09.439085+0000","last_clean":"2024-01-20T08:59:09.439085+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:59:09.439085+0000","last_undegraded":"2024-01-20T08:59:09.439085+0000","last_fullsized":"2024-01-20T08:59:09.439085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:09.438951+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:09.438951+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.438951+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:13:50.002386+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"27'655","reported_seq":694,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.257018+0000","last_change":"2024-01-20T08:59:11.257062+0000","last_active":"2024-01-20T08:59:11.257018+0000","last_peered":"2024-01-20T08:59:11.257018+0000","last_clean":"2024-01-20T08:59:11.257018+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:59:11.257018+0000","last_undegraded":"2024-01-20T08:59:11.257018+0000","last_fullsized":"2024-01-20T08:59:11.257018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'655","last_scrub_stamp":"2024-01-20T08:59:11.256922+0000","last_deep_scrub":"27'655","last_deep_scrub_stamp":"2024-01-20T08:59:11.256922+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.256922+0000","objects_scrubbed":4,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:30:33.251237+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"27'155","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.044048+0000","last_change":"2024-01-20T08:59:14.044048+0000","last_active":"2024-01-20T08:59:14.044048+0000","last_peered":"2024-01-20T08:59:14.044048+0000","last_clean":"2024-01-20T08:59:14.044048+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:59:14.044048+0000","last_undegraded":"2024-01-20T08:59:14.044048+0000","last_fullsized":"2024-01-20T08:59:14.044048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'155","last_scrub_stamp":"2024-01-20T08:59:14.043962+0000","last_deep_scrub":"27'155","last_deep_scrub_stamp":"2024-01-20T08:59:14.043962+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.043962+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:44.085163+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":23255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'39","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.059523+0000","last_change":"2024-01-20T08:59:09.059546+0000","last_active":"2024-01-20T08:59:09.059523+0000","last_peered":"2024-01-20T08:59:09.059523+0000","last_clean":"2024-01-20T08:59:09.059523+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:59:09.059523+0000","last_undegraded":"2024-01-20T08:59:09.059523+0000","last_fullsized":"2024-01-20T08:59:09.059523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:09.059430+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:09.059430+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.059430+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:59.471964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'8","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.792880+0000","last_change":"2024-01-20T08:59:10.792908+0000","last_active":"2024-01-20T08:59:10.792880+0000","last_peered":"2024-01-20T08:59:10.792880+0000","last_clean":"2024-01-20T08:59:10.792880+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:59:10.792880+0000","last_undegraded":"2024-01-20T08:59:10.792880+0000","last_fullsized":"2024-01-20T08:59:10.792880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:10.792812+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:10.792812+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.792812+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:13:12.769316+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"27'124","reported_seq":160,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.149518+0000","last_change":"2024-01-20T08:59:18.149518+0000","last_active":"2024-01-20T08:59:18.149518+0000","last_peered":"2024-01-20T08:59:18.149518+0000","last_clean":"2024-01-20T08:59:18.149518+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:59:18.149518+0000","last_undegraded":"2024-01-20T08:59:18.149518+0000","last_fullsized":"2024-01-20T08:59:18.149518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'124","last_scrub_stamp":"2024-01-20T08:59:18.149440+0000","last_deep_scrub":"27'124","last_deep_scrub_stamp":"2024-01-20T08:59:18.149440+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.149440+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:34:05.755461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"27'42","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.078143+0000","last_change":"2024-01-20T08:59:11.078180+0000","last_active":"2024-01-20T08:59:11.078143+0000","last_peered":"2024-01-20T08:59:11.078143+0000","last_clean":"2024-01-20T08:59:11.078143+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:59:11.078143+0000","last_undegraded":"2024-01-20T08:59:11.078143+0000","last_fullsized":"2024-01-20T08:59:11.078143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:11.078073+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:11.078073+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.078073+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:02:01.490607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'15","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.836288+0000","last_change":"2024-01-20T08:59:06.836330+0000","last_active":"2024-01-20T08:59:06.836288+0000","last_peered":"2024-01-20T08:59:06.836288+0000","last_clean":"2024-01-20T08:59:06.836288+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:59:06.836288+0000","last_undegraded":"2024-01-20T08:59:06.836288+0000","last_fullsized":"2024-01-20T08:59:06.836288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-01-20T08:59:06.836229+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-01-20T08:59:06.836229+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.836229+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:58:53.564126+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"27'155","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.723501+0000","last_change":"2024-01-20T08:59:19.723523+0000","last_active":"2024-01-20T08:59:19.723501+0000","last_peered":"2024-01-20T08:59:19.723501+0000","last_clean":"2024-01-20T08:59:19.723501+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:59:19.723501+0000","last_undegraded":"2024-01-20T08:59:19.723501+0000","last_fullsized":"2024-01-20T08:59:19.723501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'155","last_scrub_stamp":"2024-01-20T08:59:19.723439+0000","last_deep_scrub":"27'155","last_deep_scrub_stamp":"2024-01-20T08:59:19.723439+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.723439+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:28:39.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"27'130","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.704643+0000","last_change":"2024-01-20T08:59:22.704685+0000","last_active":"2024-01-20T08:59:22.704643+0000","last_peered":"2024-01-20T08:59:22.704643+0000","last_clean":"2024-01-20T08:59:22.704643+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:59:22.704643+0000","last_undegraded":"2024-01-20T08:59:22.704643+0000","last_fullsized":"2024-01-20T08:59:22.704643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-01-20T08:59:22.704542+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-01-20T08:59:22.704542+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.704542+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:20:09.614121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":8002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'42","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.692010+0000","last_change":"2024-01-20T08:59:17.692033+0000","last_active":"2024-01-20T08:59:17.692010+0000","last_peered":"2024-01-20T08:59:17.692010+0000","last_clean":"2024-01-20T08:59:17.692010+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:59:17.692010+0000","last_undegraded":"2024-01-20T08:59:17.692010+0000","last_fullsized":"2024-01-20T08:59:17.692010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:17.691942+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:17.691942+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.691942+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:38:46.294949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"27'8","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.666558+0000","last_change":"2024-01-20T08:59:22.666579+0000","last_active":"2024-01-20T08:59:22.666558+0000","last_peered":"2024-01-20T08:59:22.666558+0000","last_clean":"2024-01-20T08:59:22.666558+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:59:22.666558+0000","last_undegraded":"2024-01-20T08:59:22.666558+0000","last_fullsized":"2024-01-20T08:59:22.666558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:22.666499+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:22.666499+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.666499+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:25:52.125761+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"27'143","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:58:25.834086+0000","last_change":"2024-01-20T08:31:16.180216+0000","last_active":"2024-01-20T08:58:25.834086+0000","last_peered":"2024-01-20T08:58:25.834086+0000","last_clean":"2024-01-20T08:58:25.834086+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:58:25.834086+0000","last_undegraded":"2024-01-20T08:58:25.834086+0000","last_fullsized":"2024-01-20T08:58:25.834086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-20T08:31:14.848500+0000","last_clean_scrub_stamp":"2024-01-20T08:31:14.848500+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":18955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'41","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.051349+0000","last_change":"2024-01-20T08:59:23.051374+0000","last_active":"2024-01-20T08:59:23.051349+0000","last_peered":"2024-01-20T08:59:23.051349+0000","last_clean":"2024-01-20T08:59:23.051349+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:59:23.051349+0000","last_undegraded":"2024-01-20T08:59:23.051349+0000","last_fullsized":"2024-01-20T08:59:23.051349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:23.051275+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:23.051275+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.051275+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:07:58.886758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'9","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.345284+0000","last_change":"2024-01-20T08:59:10.345326+0000","last_active":"2024-01-20T08:59:10.345284+0000","last_peered":"2024-01-20T08:59:10.345284+0000","last_clean":"2024-01-20T08:59:10.345284+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:59:10.345284+0000","last_undegraded":"2024-01-20T08:59:10.345284+0000","last_fullsized":"2024-01-20T08:59:10.345284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-01-20T08:59:10.345200+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-01-20T08:59:10.345200+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.345200+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:49:06.776395+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"27'679","reported_seq":718,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.614378+0000","last_change":"2024-01-20T08:59:17.614404+0000","last_active":"2024-01-20T08:59:17.614378+0000","last_peered":"2024-01-20T08:59:17.614378+0000","last_clean":"2024-01-20T08:59:17.614378+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:59:17.614378+0000","last_undegraded":"2024-01-20T08:59:17.614378+0000","last_fullsized":"2024-01-20T08:59:17.614378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'679","last_scrub_stamp":"2024-01-20T08:59:17.614324+0000","last_deep_scrub":"27'679","last_deep_scrub_stamp":"2024-01-20T08:59:17.614324+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.614324+0000","objects_scrubbed":4,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:33:59.859639+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'39","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.742148+0000","last_change":"2024-01-20T08:59:15.742170+0000","last_active":"2024-01-20T08:59:15.742148+0000","last_peered":"2024-01-20T08:59:15.742148+0000","last_clean":"2024-01-20T08:59:15.742148+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:59:15.742148+0000","last_undegraded":"2024-01-20T08:59:15.742148+0000","last_fullsized":"2024-01-20T08:59:15.742148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:15.742087+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:15.742087+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.742087+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:29:59.469179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"27'89","reported_seq":124,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.735493+0000","last_change":"2024-01-20T08:59:21.735538+0000","last_active":"2024-01-20T08:59:21.735493+0000","last_peered":"2024-01-20T08:59:21.735493+0000","last_clean":"2024-01-20T08:59:21.735493+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:59:21.735493+0000","last_undegraded":"2024-01-20T08:59:21.735493+0000","last_fullsized":"2024-01-20T08:59:21.735493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'89","last_scrub_stamp":"2024-01-20T08:59:21.735407+0000","last_deep_scrub":"27'89","last_deep_scrub_stamp":"2024-01-20T08:59:21.735407+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.735407+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:38.010221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":9333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'691","reported_seq":730,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.000879+0000","last_change":"2024-01-20T08:59:13.000926+0000","last_active":"2024-01-20T08:59:13.000879+0000","last_peered":"2024-01-20T08:59:13.000879+0000","last_clean":"2024-01-20T08:59:13.000879+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:59:13.000879+0000","last_undegraded":"2024-01-20T08:59:13.000879+0000","last_fullsized":"2024-01-20T08:59:13.000879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'691","last_scrub_stamp":"2024-01-20T08:59:13.000787+0000","last_deep_scrub":"27'691","last_deep_scrub_stamp":"2024-01-20T08:59:13.000787+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.000787+0000","objects_scrubbed":3,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:23.559691+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"27'33","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.805565+0000","last_change":"2024-01-20T08:59:08.805605+0000","last_active":"2024-01-20T08:59:08.805565+0000","last_peered":"2024-01-20T08:59:08.805565+0000","last_clean":"2024-01-20T08:59:08.805565+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:59:08.805565+0000","last_undegraded":"2024-01-20T08:59:08.805565+0000","last_fullsized":"2024-01-20T08:59:08.805565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:08.805473+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:08.805473+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.805473+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:35.530260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"27'147","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.146924+0000","last_change":"2024-01-20T08:59:17.146946+0000","last_active":"2024-01-20T08:59:17.146924+0000","last_peered":"2024-01-20T08:59:17.146924+0000","last_clean":"2024-01-20T08:59:17.146924+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:59:17.146924+0000","last_undegraded":"2024-01-20T08:59:17.146924+0000","last_fullsized":"2024-01-20T08:59:17.146924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'147","last_scrub_stamp":"2024-01-20T08:59:17.146851+0000","last_deep_scrub":"27'147","last_deep_scrub_stamp":"2024-01-20T08:59:17.146851+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.146851+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:50:24.411608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":10430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"27'50","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.087872+0000","last_change":"2024-01-20T08:59:12.087893+0000","last_active":"2024-01-20T08:59:12.087872+0000","last_peered":"2024-01-20T08:59:12.087872+0000","last_clean":"2024-01-20T08:59:12.087872+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:59:12.087872+0000","last_undegraded":"2024-01-20T08:59:12.087872+0000","last_fullsized":"2024-01-20T08:59:12.087872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-01-20T08:59:12.087798+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-01-20T08:59:12.087798+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.087798+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:26.121715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"27'50","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.252254+0000","last_change":"2024-01-20T08:59:14.252274+0000","last_active":"2024-01-20T08:59:14.252254+0000","last_peered":"2024-01-20T08:59:14.252254+0000","last_clean":"2024-01-20T08:59:14.252254+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:59:14.252254+0000","last_undegraded":"2024-01-20T08:59:14.252254+0000","last_fullsized":"2024-01-20T08:59:14.252254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-01-20T08:59:14.252192+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-01-20T08:59:14.252192+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.252192+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:47.209255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"27'44","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.500933+0000","last_change":"2024-01-20T08:59:14.500953+0000","last_active":"2024-01-20T08:59:14.500933+0000","last_peered":"2024-01-20T08:59:14.500933+0000","last_clean":"2024-01-20T08:59:14.500933+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:59:14.500933+0000","last_undegraded":"2024-01-20T08:59:14.500933+0000","last_fullsized":"2024-01-20T08:59:14.500933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'44","last_scrub_stamp":"2024-01-20T08:59:14.500872+0000","last_deep_scrub":"27'44","last_deep_scrub_stamp":"2024-01-20T08:59:14.500872+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.500872+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:10:40.652145+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'35","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.987389+0000","last_change":"2024-01-20T08:59:18.987428+0000","last_active":"2024-01-20T08:59:18.987389+0000","last_peered":"2024-01-20T08:59:18.987389+0000","last_clean":"2024-01-20T08:59:18.987389+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:59:18.987389+0000","last_undegraded":"2024-01-20T08:59:18.987389+0000","last_fullsized":"2024-01-20T08:59:18.987389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:18.987295+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:18.987295+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.987295+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:47:38.830913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"27'27","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.371846+0000","last_change":"2024-01-20T08:59:16.371846+0000","last_active":"2024-01-20T08:59:16.371846+0000","last_peered":"2024-01-20T08:59:16.371846+0000","last_clean":"2024-01-20T08:59:16.371846+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:59:16.371846+0000","last_undegraded":"2024-01-20T08:59:16.371846+0000","last_fullsized":"2024-01-20T08:59:16.371846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-01-20T08:59:16.371699+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-01-20T08:59:16.371699+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.371699+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:13:18.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'31","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.349087+0000","last_change":"2024-01-20T08:59:12.349087+0000","last_active":"2024-01-20T08:59:12.349087+0000","last_peered":"2024-01-20T08:59:12.349087+0000","last_clean":"2024-01-20T08:59:12.349087+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:59:12.349087+0000","last_undegraded":"2024-01-20T08:59:12.349087+0000","last_fullsized":"2024-01-20T08:59:12.349087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:12.349009+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:12.349009+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.349009+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:42:47.241016+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"27'44","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.029595+0000","last_change":"2024-01-20T08:59:10.029595+0000","last_active":"2024-01-20T08:59:10.029595+0000","last_peered":"2024-01-20T08:59:10.029595+0000","last_clean":"2024-01-20T08:59:10.029595+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:59:10.029595+0000","last_undegraded":"2024-01-20T08:59:10.029595+0000","last_fullsized":"2024-01-20T08:59:10.029595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'44","last_scrub_stamp":"2024-01-20T08:59:10.029487+0000","last_deep_scrub":"27'44","last_deep_scrub_stamp":"2024-01-20T08:59:10.029487+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.029487+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:37:12.065884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"27'34","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.992797+0000","last_change":"2024-01-20T08:59:20.992797+0000","last_active":"2024-01-20T08:59:20.992797+0000","last_peered":"2024-01-20T08:59:20.992797+0000","last_clean":"2024-01-20T08:59:20.992797+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:59:20.992797+0000","last_undegraded":"2024-01-20T08:59:20.992797+0000","last_fullsized":"2024-01-20T08:59:20.992797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:20.992690+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:20.992690+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.992690+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:15:39.303881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"27'34","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:24.718885+0000","last_change":"2024-01-20T08:59:24.718929+0000","last_active":"2024-01-20T08:59:24.718885+0000","last_peered":"2024-01-20T08:59:24.718885+0000","last_clean":"2024-01-20T08:59:24.718885+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:59:24.718885+0000","last_undegraded":"2024-01-20T08:59:24.718885+0000","last_fullsized":"2024-01-20T08:59:24.718885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:24.718791+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:24.718791+0000","last_clean_scrub_stamp":"2024-01-20T08:59:24.718791+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:11:24.369143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"27'42","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.048893+0000","last_change":"2024-01-20T08:59:17.048937+0000","last_active":"2024-01-20T08:59:17.048893+0000","last_peered":"2024-01-20T08:59:17.048893+0000","last_clean":"2024-01-20T08:59:17.048893+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:59:17.048893+0000","last_undegraded":"2024-01-20T08:59:17.048893+0000","last_fullsized":"2024-01-20T08:59:17.048893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:17.048777+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:17.048777+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.048777+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:02:46.925145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"27'38","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.894238+0000","last_change":"2024-01-20T08:59:11.894275+0000","last_active":"2024-01-20T08:59:11.894238+0000","last_peered":"2024-01-20T08:59:11.894238+0000","last_clean":"2024-01-20T08:59:11.894238+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:59:11.894238+0000","last_undegraded":"2024-01-20T08:59:11.894238+0000","last_fullsized":"2024-01-20T08:59:11.894238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'38","last_scrub_stamp":"2024-01-20T08:59:11.894159+0000","last_deep_scrub":"27'38","last_deep_scrub_stamp":"2024-01-20T08:59:11.894159+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.894159+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:23:22.695818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'47","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.725765+0000","last_change":"2024-01-20T08:59:13.725788+0000","last_active":"2024-01-20T08:59:13.725765+0000","last_peered":"2024-01-20T08:59:13.725765+0000","last_clean":"2024-01-20T08:59:13.725765+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:59:13.725765+0000","last_undegraded":"2024-01-20T08:59:13.725765+0000","last_fullsized":"2024-01-20T08:59:13.725765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-01-20T08:59:13.725705+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-01-20T08:59:13.725705+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.725705+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:43.766906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"27'41","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.702383+0000","last_change":"2024-01-20T08:59:17.702422+0000","last_active":"2024-01-20T08:59:17.702383+0000","last_peered":"2024-01-20T08:59:17.702383+0000","last_clean":"2024-01-20T08:59:17.702383+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:59:17.702383+0000","last_undegraded":"2024-01-20T08:59:17.702383+0000","last_fullsized":"2024-01-20T08:59:17.702383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:17.702301+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:17.702301+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.702301+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:33.977115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"27'34","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.463984+0000","last_change":"2024-01-20T08:59:10.464006+0000","last_active":"2024-01-20T08:59:10.463984+0000","last_peered":"2024-01-20T08:59:10.463984+0000","last_clean":"2024-01-20T08:59:10.463984+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:59:10.463984+0000","last_undegraded":"2024-01-20T08:59:10.463984+0000","last_fullsized":"2024-01-20T08:59:10.463984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:10.463920+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:10.463920+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.463920+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:31:57.709584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"27'45","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.255978+0000","last_change":"2024-01-20T08:59:13.256000+0000","last_active":"2024-01-20T08:59:13.255978+0000","last_peered":"2024-01-20T08:59:13.255978+0000","last_clean":"2024-01-20T08:59:13.255978+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:59:13.255978+0000","last_undegraded":"2024-01-20T08:59:13.255978+0000","last_fullsized":"2024-01-20T08:59:13.255978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-01-20T08:59:13.255913+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-01-20T08:59:13.255913+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.255913+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:17.393393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"27'38","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.446007+0000","last_change":"2024-01-20T08:59:06.446007+0000","last_active":"2024-01-20T08:59:06.446007+0000","last_peered":"2024-01-20T08:59:06.446007+0000","last_clean":"2024-01-20T08:59:06.446007+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:59:06.446007+0000","last_undegraded":"2024-01-20T08:59:06.446007+0000","last_fullsized":"2024-01-20T08:59:06.446007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'38","last_scrub_stamp":"2024-01-20T08:59:06.445911+0000","last_deep_scrub":"27'38","last_deep_scrub_stamp":"2024-01-20T08:59:06.445911+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.445911+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:32:02.791907+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"27'42","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.491798+0000","last_change":"2024-01-20T08:59:11.491823+0000","last_active":"2024-01-20T08:59:11.491798+0000","last_peered":"2024-01-20T08:59:11.491798+0000","last_clean":"2024-01-20T08:59:11.491798+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:59:11.491798+0000","last_undegraded":"2024-01-20T08:59:11.491798+0000","last_fullsized":"2024-01-20T08:59:11.491798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:11.491731+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:11.491731+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.491731+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:26:49.661101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"27'39","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.048545+0000","last_change":"2024-01-20T08:59:16.048566+0000","last_active":"2024-01-20T08:59:16.048545+0000","last_peered":"2024-01-20T08:59:16.048545+0000","last_clean":"2024-01-20T08:59:16.048545+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:59:16.048545+0000","last_undegraded":"2024-01-20T08:59:16.048545+0000","last_fullsized":"2024-01-20T08:59:16.048545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:16.048485+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:16.048485+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.048485+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:47.482687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"27'36","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.389145+0000","last_change":"2024-01-20T08:59:14.389145+0000","last_active":"2024-01-20T08:59:14.389145+0000","last_peered":"2024-01-20T08:59:14.389145+0000","last_clean":"2024-01-20T08:59:14.389145+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:59:14.389145+0000","last_undegraded":"2024-01-20T08:59:14.389145+0000","last_fullsized":"2024-01-20T08:59:14.389145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:14.389061+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:14.389061+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.389061+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:56.661649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"27'37","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.354770+0000","last_change":"2024-01-20T08:59:18.354808+0000","last_active":"2024-01-20T08:59:18.354770+0000","last_peered":"2024-01-20T08:59:18.354770+0000","last_clean":"2024-01-20T08:59:18.354770+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:59:18.354770+0000","last_undegraded":"2024-01-20T08:59:18.354770+0000","last_fullsized":"2024-01-20T08:59:18.354770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:18.354671+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:18.354671+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.354671+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:04:39.530834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'28","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.742962+0000","last_change":"2024-01-20T08:59:18.742983+0000","last_active":"2024-01-20T08:59:18.742962+0000","last_peered":"2024-01-20T08:59:18.742962+0000","last_clean":"2024-01-20T08:59:18.742962+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:59:18.742962+0000","last_undegraded":"2024-01-20T08:59:18.742962+0000","last_fullsized":"2024-01-20T08:59:18.742962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-01-20T08:59:18.742901+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-01-20T08:59:18.742901+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.742901+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:20:05.652480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"27'31","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.800935+0000","last_change":"2024-01-20T08:59:14.800956+0000","last_active":"2024-01-20T08:59:14.800935+0000","last_peered":"2024-01-20T08:59:14.800935+0000","last_clean":"2024-01-20T08:59:14.800935+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:59:14.800935+0000","last_undegraded":"2024-01-20T08:59:14.800935+0000","last_fullsized":"2024-01-20T08:59:14.800935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:14.800871+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:14.800871+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.800871+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:59:50.527407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"27'48","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.790132+0000","last_change":"2024-01-20T08:59:12.790168+0000","last_active":"2024-01-20T08:59:12.790132+0000","last_peered":"2024-01-20T08:59:12.790132+0000","last_clean":"2024-01-20T08:59:12.790132+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:59:12.790132+0000","last_undegraded":"2024-01-20T08:59:12.790132+0000","last_fullsized":"2024-01-20T08:59:12.790132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-01-20T08:59:12.790039+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-01-20T08:59:12.790039+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.790039+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:04:33.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"27'37","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.870735+0000","last_change":"2024-01-20T08:59:10.870757+0000","last_active":"2024-01-20T08:59:10.870735+0000","last_peered":"2024-01-20T08:59:10.870735+0000","last_clean":"2024-01-20T08:59:10.870735+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:59:10.870735+0000","last_undegraded":"2024-01-20T08:59:10.870735+0000","last_fullsized":"2024-01-20T08:59:10.870735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:10.870674+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:10.870674+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.870674+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:32:28.888081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"27'29","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.056411+0000","last_change":"2024-01-20T08:59:10.056470+0000","last_active":"2024-01-20T08:59:10.056411+0000","last_peered":"2024-01-20T08:59:10.056411+0000","last_clean":"2024-01-20T08:59:10.056411+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:59:10.056411+0000","last_undegraded":"2024-01-20T08:59:10.056411+0000","last_fullsized":"2024-01-20T08:59:10.056411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-01-20T08:59:10.056323+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-01-20T08:59:10.056323+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.056323+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:24.090240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'36","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:05.751587+0000","last_change":"2024-01-20T08:59:05.751608+0000","last_active":"2024-01-20T08:59:05.751587+0000","last_peered":"2024-01-20T08:59:05.751587+0000","last_clean":"2024-01-20T08:59:05.751587+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:59:05.751587+0000","last_undegraded":"2024-01-20T08:59:05.751587+0000","last_fullsized":"2024-01-20T08:59:05.751587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:05.751528+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:05.751528+0000","last_clean_scrub_stamp":"2024-01-20T08:59:05.751528+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:40:23.102642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"27'37","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.081918+0000","last_change":"2024-01-20T08:59:08.081958+0000","last_active":"2024-01-20T08:59:08.081918+0000","last_peered":"2024-01-20T08:59:08.081918+0000","last_clean":"2024-01-20T08:59:08.081918+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:59:08.081918+0000","last_undegraded":"2024-01-20T08:59:08.081918+0000","last_fullsized":"2024-01-20T08:59:08.081918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:08.081835+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:08.081835+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.081835+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:23:18.883494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"27'30","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.726790+0000","last_change":"2024-01-20T08:59:16.726824+0000","last_active":"2024-01-20T08:59:16.726790+0000","last_peered":"2024-01-20T08:59:16.726790+0000","last_clean":"2024-01-20T08:59:16.726790+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:59:16.726790+0000","last_undegraded":"2024-01-20T08:59:16.726790+0000","last_fullsized":"2024-01-20T08:59:16.726790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-01-20T08:59:16.726728+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-01-20T08:59:16.726728+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.726728+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:46.998792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.101535+0000","last_change":"2024-01-20T08:59:07.101535+0000","last_active":"2024-01-20T08:59:07.101535+0000","last_peered":"2024-01-20T08:59:07.101535+0000","last_clean":"2024-01-20T08:59:07.101535+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:59:07.101535+0000","last_undegraded":"2024-01-20T08:59:07.101535+0000","last_fullsized":"2024-01-20T08:59:07.101535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-01-20T08:59:07.101443+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-01-20T08:59:07.101443+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.101443+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:32:25.118851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'41","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.008640+0000","last_change":"2024-01-20T08:59:11.008662+0000","last_active":"2024-01-20T08:59:11.008640+0000","last_peered":"2024-01-20T08:59:11.008640+0000","last_clean":"2024-01-20T08:59:11.008640+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:59:11.008640+0000","last_undegraded":"2024-01-20T08:59:11.008640+0000","last_fullsized":"2024-01-20T08:59:11.008640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:11.008572+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:11.008572+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.008572+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:53.281160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"27'33","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.013088+0000","last_change":"2024-01-20T08:59:22.013111+0000","last_active":"2024-01-20T08:59:22.013088+0000","last_peered":"2024-01-20T08:59:22.013088+0000","last_clean":"2024-01-20T08:59:22.013088+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:59:22.013088+0000","last_undegraded":"2024-01-20T08:59:22.013088+0000","last_fullsized":"2024-01-20T08:59:22.013088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:22.013017+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:22.013017+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.013017+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:18:30.736742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"27'26","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:04.749273+0000","last_change":"2024-01-20T08:59:04.749310+0000","last_active":"2024-01-20T08:59:04.749273+0000","last_peered":"2024-01-20T08:59:04.749273+0000","last_clean":"2024-01-20T08:59:04.749273+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:59:04.749273+0000","last_undegraded":"2024-01-20T08:59:04.749273+0000","last_fullsized":"2024-01-20T08:59:04.749273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-01-20T08:59:04.749186+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-01-20T08:59:04.749186+0000","last_clean_scrub_stamp":"2024-01-20T08:59:04.749186+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:15.202067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":308552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4420,"ondisk_log_size":4420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39037201,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":8593,"num_write_kb":43222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78323712,"data_stored":78074402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8212,"ondisk_log_size":8212,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509790,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39732,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332108,"statfs":{"total":96636764160,"available":96596078592,"internally_reserved":0,"allocated":4501504,"data_stored":4315319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.39600000000000002,"5min":0.29199999999999998,"15min":0.23899999999999999},"max":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.67100000000000004},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.81999999999999995},"last":0.44400000000000001}]},{"osd":1,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.35299999999999998,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.67900000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.28999999999999998},"max":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.67500000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.42099999999999999,"5min":0.30099999999999999,"15min":0.28399999999999997},"max":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.70899999999999996},"last":0.37},{"interface":"front","average":{"1min":0.5,"5min":0.46500000000000002,"15min":0.46999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.629,"5min":0.68500000000000005,"15min":0.72699999999999998},"last":0.40100000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.34999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.60899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30299999999999999},"max":{"1min":0.56799999999999995,"5min":0.629,"15min":0.74199999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.312,"5min":0.248,"15min":0.23799999999999999},"max":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.76700000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.30399999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67200000000000004,"5min":0.75800000000000001,"15min":0.876},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.439,"15min":0.45200000000000001},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58199999999999996,"5min":0.67400000000000004,"15min":0.81000000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68700000000000006,"5min":0.73299999999999998,"15min":0.79400000000000004},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.62,"5min":0.78400000000000003,"15min":0.90900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.50600000000000001,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.85299999999999998},"last":0.60299999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509790,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67036,"kb_used_data":13652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94304804,"statfs":{"total":96636764160,"available":96568119296,"internally_reserved":0,"allocated":13979648,"data_stored":13773597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.29099999999999998},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17199999999999999},"max":{"1min":0.41899999999999998,"5min":0.441,"15min":0.72599999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.19500000000000001,"15min":0.19400000000000001},"max":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.58799999999999997},"last":0.318}]},{"osd":2,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.503,"15min":0.61799999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"last":0.28699999999999998}]},{"osd":3,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.216,"15min":0.21099999999999999},"max":{"1min":0.39500000000000002,"5min":0.51000000000000001,"15min":0.79600000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.69499999999999995},"last":0.34200000000000003}]},{"osd":4,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.72599999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80100000000000005},"last":0.5},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.77600000000000002,"15min":0.91400000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.53600000000000003}]},{"osd":7,"last update":"Sat Jan 20 08:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.78500000000000003},"last":0.54700000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542492,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70924,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300916,"statfs":{"total":96636764160,"available":96564137984,"internally_reserved":0,"allocated":13701120,"data_stored":13502280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"max":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.69599999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72099999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.245},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.66000000000000003},"last":0.315}]},{"osd":2,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.72899999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.379}]},{"osd":3,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.76700000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.80900000000000005},"last":0.37}]},{"osd":4,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44,"15min":0.439},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48399999999999999,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.26500000000000001},"max":{"1min":0.56799999999999995,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.81599999999999995},"last":0.41699999999999998}]},{"osd":7,"last update":"Sat Jan 20 08:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.80300000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59399999999999997,"5min":0.76200000000000001,"15min":0.85799999999999998},"last":0.442}]}]},{"osd":4,"up_from":14,"seq":60129542491,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66068,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305772,"statfs":{"total":96636764160,"available":96569110528,"internally_reserved":0,"allocated":8728576,"data_stored":8508591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.61299999999999999,"15min":0.70199999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.59999999999999998}]},{"osd":1,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.629,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54900000000000004}]},{"osd":2,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.31,"15min":0.26500000000000001},"max":{"1min":0.51400000000000001,"5min":0.69599999999999995,"15min":0.73299999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.315,"5min":0.31,"15min":0.28599999999999998},"max":{"1min":0.54700000000000004,"5min":0.622,"15min":0.66700000000000004},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57799999999999996,"5min":0.71899999999999997,"15min":0.73999999999999999},"last":0.437},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.254},"max":{"1min":0.61899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.59099999999999997,"5min":0.75700000000000001,"15min":0.78400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":0.75600000000000001},"last":0.498}]},{"osd":7,"last update":"Sat Jan 20 08:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66000000000000003,"5min":0.78300000000000003,"15min":0.79100000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":0.71699999999999997,"15min":0.79000000000000004},"last":0.51200000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542492,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43824,"kb_used_data":8488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328016,"statfs":{"total":96636764160,"available":96591888384,"internally_reserved":0,"allocated":8691712,"data_stored":8506821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.45800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.309},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29999999999999999,"15min":0.307},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":0.38100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.26800000000000002}]},{"osd":1,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.502,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.34999999999999998}]},{"osd":3,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33000000000000002,"15min":0.33200000000000002},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.378,"5min":0.55900000000000005,"15min":0.60799999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41099999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.71399999999999997,"15min":0.82099999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.86699999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Jan 20 08:58:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.80500000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.69999999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"last":0.65400000000000003}]},{"osd":7,"last update":"Sat Jan 20 08:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.76000000000000001,"15min":0.79300000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.36399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85299999999999998},"last":0.73399999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542493,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9289728,"data_stored":9097863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30099999999999999,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.55700000000000005,"15min":0.59499999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.32000000000000001,"15min":0.31},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20000000000000001},"max":{"1min":0.40600000000000003,"5min":0.45500000000000002,"15min":0.54400000000000004},"last":0.28299999999999997}]},{"osd":2,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.308,"15min":0.314},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.35399999999999998,"5min":0.47999999999999998,"15min":0.49199999999999999},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":0.45800000000000002,"5min":0.624,"15min":0.624},"last":0.33900000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.59399999999999997,"15min":0.63},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.44700000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.255}]},{"osd":4,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.27900000000000003},"max":{"1min":0.80700000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.38300000000000001}]},{"osd":5,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.876},"last":0.373}]},{"osd":6,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":0.873,"15min":0.873},"last":0.41199999999999998}]},{"osd":7,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.43099999999999999}]}]},{"osd":7,"up_from":13,"seq":55834575196,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63212,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308628,"statfs":{"total":96636764160,"available":96572035072,"internally_reserved":0,"allocated":5804032,"data_stored":5592459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.373},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"last":0.51900000000000002}]},{"osd":2,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.68999999999999995},"last":0.40200000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.54200000000000004,"5min":0.65100000000000002,"15min":0.73499999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.37,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.78000000000000003},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23000000000000001},"max":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.75800000000000001},"last":0.433},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26600000000000001},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.79900000000000004},"last":0.44400000000000001}]},{"osd":5,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.76400000000000001,"15min":0.77800000000000002},"last":0.38500000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.498},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.50800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575196,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61772,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310068,"statfs":{"total":96636764160,"available":96573509632,"internally_reserved":0,"allocated":17108992,"data_stored":16897992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.505,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.31,"15min":0.30399999999999999},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.41699999999999998,"5min":0.47599999999999998,"15min":0.51200000000000001},"last":0.215}]},{"osd":1,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47199999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.42299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44700000000000001,"5min":0.625,"15min":0.625},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.373,"15min":0.371},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.434,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35599999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.78500000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.71399999999999997,"5min":0.80000000000000004,"15min":0.85499999999999998},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.76100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.34000000000000002},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13692928,"data_stored":13656124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8409088,"data_stored":8390022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":8389348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16822272,"data_stored":16780519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8441856,"data_stored":8391118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12820480,"data_stored":12794439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4214784,"data_stored":4197846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":5474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:59:30.768 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:30.768 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:30.769 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:30.769 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-01-20T08:31:14.848500+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=31, tm_sec=14, tm_wday=5, tm_yday=20, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=20, tm_hour=8, tm_min=59, tm_sec=2, tm_wday=5, tm_yday=20, tm_isdst=0) 2024-01-20T08:59:30.770 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-20T08:59:46.590 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:59:46.592 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T08:59:50.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-20T08:59:50.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.870+0000 7f664d37c640 1 Processor -- start 2024-01-20T08:59:50.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.870+0000 7f664d37c640 1 -- start start 2024-01-20T08:59:50.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.871+0000 7f664d37c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66481573e0 0x7f66480c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:50.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.871+0000 7f664d37c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66480c2930 con 0x7f66481518e0 2024-01-20T08:59:50.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.871+0000 7f664d37c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6648157c20 con 0x7f66480c38c0 2024-01-20T08:59:50.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.871+0000 7f664d37c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66480c85c0 con 0x7f66481573e0 2024-01-20T08:59:50.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.872+0000 7f66467fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66481573e0 0x7f66480c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:50.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.873+0000 7f6646ffd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f66481518e0 0x7f66480c7bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51376/0 (socket says 172.21.15.87:51376) 2024-01-20T08:59:50.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.873+0000 7f6646ffd640 1 -- 172.21.15.87:0/3792512129 learned_addr learned my addr 172.21.15.87:0/3792512129 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:50.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.875+0000 7f66467fc640 1 -- 172.21.15.87:0/3792512129 >> v1:172.21.15.116:6789/0 conn(0x7f66480c38c0 legacy=0x7f66480ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:50.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.875+0000 7f66467fc640 1 -- 172.21.15.87:0/3792512129 >> v1:172.21.15.87:6789/0 conn(0x7f66481518e0 legacy=0x7f66480c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:50.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.875+0000 7f66467fc640 1 -- 172.21.15.87:0/3792512129 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6648157f00 con 0x7f66481573e0 2024-01-20T08:59:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.879+0000 7f66467fc640 1 --2- 172.21.15.87:0/3792512129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f66480c3380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f663c0097a0 tx=0x7f663c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0634837aac319d server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:50.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.883+0000 7f6645ffb640 1 -- 172.21.15.87:0/3792512129 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c02f810 con 0x7f66481573e0 2024-01-20T08:59:50.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.883+0000 7f6645ffb640 1 -- 172.21.15.87:0/3792512129 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c02f970 con 0x7f66481573e0 2024-01-20T08:59:50.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.883+0000 7f6645ffb640 1 -- 172.21.15.87:0/3792512129 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c02fd00 con 0x7f66481573e0 2024-01-20T08:59:50.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 -- 172.21.15.87:0/3792512129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 msgr2=0x7f66480c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:50.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 --2- 172.21.15.87:0/3792512129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f66480c3380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f663c0097a0 tx=0x7f663c02b9f0 comp rx=0 tx=0).stop 2024-01-20T08:59:50.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 -- 172.21.15.87:0/3792512129 shutdown_connections 2024-01-20T08:59:50.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 --2- 172.21.15.87:0/3792512129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f66480c3380 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:50.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 -- 172.21.15.87:0/3792512129 >> 172.21.15.87:0/3792512129 conn(0x7f664806ef10 msgr2=0x7f664806f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:50.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 -- 172.21.15.87:0/3792512129 shutdown_connections 2024-01-20T08:59:50.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.884+0000 7f664d37c640 1 -- 172.21.15.87:0/3792512129 wait complete. 2024-01-20T08:59:50.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.885+0000 7f664d37c640 1 Processor -- start 2024-01-20T08:59:50.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 -- start start 2024-01-20T08:59:50.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f66480c38c0 0x7f6648147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:50.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66481518e0 0x7f66481f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:50.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f664813e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:50.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66480ad900 con 0x7f66481573e0 2024-01-20T08:59:50.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66480ad780 con 0x7f66481518e0 2024-01-20T08:59:50.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.886+0000 7f664d37c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66480b3020 con 0x7f66480c38c0 2024-01-20T08:59:50.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.887+0000 7f66477fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f664813e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.887+0000 7f66467fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66481518e0 0x7f66481f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.887+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f66480c38c0 0x7f6648147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.887+0000 7f66477fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f664813e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38356/0 (socket says 172.21.15.87:38356) 2024-01-20T08:59:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.887+0000 7f66477fe640 1 -- 172.21.15.87:0/3344354549 learned_addr learned my addr 172.21.15.87:0/3344354549 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.891+0000 7f66477fe640 1 -- 172.21.15.87:0/3344354549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f66480c38c0 msgr2=0x7f6648147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.891+0000 7f66477fe640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f66480c38c0 0x7f6648147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.891+0000 7f66477fe640 1 -- 172.21.15.87:0/3344354549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66481518e0 msgr2=0x7f66481f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.891+0000 7f66477fe640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66481518e0 0x7f66481f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.891+0000 7f66477fe640 1 -- 172.21.15.87:0/3344354549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c009400 con 0x7f66481573e0 2024-01-20T08:59:50.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.893+0000 7f66477fe640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f664813e560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6638001eb0 tx=0x7f663800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.897+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66380052d0 con 0x7f66481573e0 2024-01-20T08:59:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.897+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6638005430 con 0x7f66481573e0 2024-01-20T08:59:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.897+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664806cf60 con 0x7f66481573e0 2024-01-20T08:59:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.897+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638003810 con 0x7f66481573e0 2024-01-20T08:59:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.897+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664806d4e0 con 0x7f66481573e0 2024-01-20T08:59:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.898+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664806d6d0 con 0x7f66481573e0 2024-01-20T08:59:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.915+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6638003970 con 0x7f66481573e0 2024-01-20T08:59:50.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.916+0000 7f662f7fe640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f663406af00 0x7f663406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:50.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.916+0000 7f6646ffd640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f663406af00 0x7f663406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:50.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.916+0000 7f6646ffd640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f663406af00 0x7f663406d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66400012e0 tx=0x7f6640001a30 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:50.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:50.920+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f6638057be0 con 0x7f66481573e0 2024-01-20T08:59:51.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.149+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6638053f20 con 0x7f66481573e0 2024-01-20T08:59:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.318+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 --> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66480c2e60 con 0x7f663406af00 2024-01-20T08:59:51.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.327+0000 7f662f7fe640 1 -- 172.21.15.87:0/3344354549 <== mgr.4154 v2:172.21.15.87:6832/2055452307 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371919 (secure 0 0 0) 0x7f6640001f60 con 0x7f663406af00 2024-01-20T08:59:51.332 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:51.334 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-01-20T08:59:51.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f663406af00 msgr2=0x7f663406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f663406af00 0x7f663406d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66400012e0 tx=0x7f6640001a30 comp rx=0 tx=0).stop 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 msgr2=0x7f664813e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f664813e560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6638001eb0 tx=0x7f663800c890 comp rx=0 tx=0).stop 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 shutdown_connections 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f663406af00 0x7f663406d3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66481573e0 0x7f664813e560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66481518e0 0x7f66481f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 --2- 172.21.15.87:0/3344354549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f66480c38c0 0x7f6648147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 >> 172.21.15.87:0/3344354549 conn(0x7f664806ef10 msgr2=0x7f6648155910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 shutdown_connections 2024-01-20T08:59:51.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.330+0000 7f664d37c640 1 -- 172.21.15.87:0/3344354549 wait complete. 2024-01-20T08:59:51.343 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":891,"stamp":"2024-01-20T08:59:50.665199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39627569,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":1083,"num_read_kb":1128,"num_write":12670,"num_write_kb":354190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15074,"ondisk_log_size":15074,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461136,"kb_used_data":79888,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754513584,"statfs":{"total":773094113280,"available":772621910016,"internally_reserved":0,"allocated":81805312,"data_stored":80194922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390319664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"2.1f","version":"27'616","reported_seq":657,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.707135+0000","last_change":"2024-01-20T08:59:11.707161+0000","last_active":"2024-01-20T08:59:11.707135+0000","last_peered":"2024-01-20T08:59:11.707135+0000","last_clean":"2024-01-20T08:59:11.707135+0000","last_became_active":"2024-01-20T08:31:08.592922+0000","last_became_peered":"2024-01-20T08:31:08.592922+0000","last_unstale":"2024-01-20T08:59:11.707135+0000","last_undegraded":"2024-01-20T08:59:11.707135+0000","last_fullsized":"2024-01-20T08:59:11.707135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'616","last_scrub_stamp":"2024-01-20T08:59:11.707082+0000","last_deep_scrub":"27'616","last_deep_scrub_stamp":"2024-01-20T08:59:11.707082+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.707082+0000","objects_scrubbed":7,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:25.740999+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"27'27","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.005143+0000","last_change":"2024-01-20T08:59:18.005164+0000","last_active":"2024-01-20T08:59:18.005143+0000","last_peered":"2024-01-20T08:59:18.005143+0000","last_clean":"2024-01-20T08:59:18.005143+0000","last_became_active":"2024-01-20T08:31:10.259888+0000","last_became_peered":"2024-01-20T08:31:10.259888+0000","last_unstale":"2024-01-20T08:59:18.005143+0000","last_undegraded":"2024-01-20T08:59:18.005143+0000","last_fullsized":"2024-01-20T08:59:18.005143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-01-20T08:59:18.005080+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-01-20T08:59:18.005080+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.005080+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:43:18.802602+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"27'162","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.735639+0000","last_change":"2024-01-20T08:59:21.735661+0000","last_active":"2024-01-20T08:59:21.735639+0000","last_peered":"2024-01-20T08:59:21.735639+0000","last_clean":"2024-01-20T08:59:21.735639+0000","last_became_active":"2024-01-20T08:31:44.811108+0000","last_became_peered":"2024-01-20T08:31:44.811108+0000","last_unstale":"2024-01-20T08:59:21.735639+0000","last_undegraded":"2024-01-20T08:59:21.735639+0000","last_fullsized":"2024-01-20T08:59:21.735639+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'162","last_scrub_stamp":"2024-01-20T08:59:21.735574+0000","last_deep_scrub":"27'162","last_deep_scrub_stamp":"2024-01-20T08:59:21.735574+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.735574+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:23:33.326267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":10773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4,1],"acting":[0,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'697","reported_seq":739,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.037427+0000","last_change":"2024-01-20T08:59:14.037427+0000","last_active":"2024-01-20T08:59:14.037427+0000","last_peered":"2024-01-20T08:59:14.037427+0000","last_clean":"2024-01-20T08:59:14.037427+0000","last_became_active":"2024-01-20T08:31:08.595709+0000","last_became_peered":"2024-01-20T08:31:08.595709+0000","last_unstale":"2024-01-20T08:59:14.037427+0000","last_undegraded":"2024-01-20T08:59:14.037427+0000","last_fullsized":"2024-01-20T08:59:14.037427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'697","last_scrub_stamp":"2024-01-20T08:59:14.037353+0000","last_deep_scrub":"27'697","last_deep_scrub_stamp":"2024-01-20T08:59:14.037353+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.037353+0000","objects_scrubbed":1,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:00:11.324134+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"27'37","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.991343+0000","last_change":"2024-01-20T08:59:19.991382+0000","last_active":"2024-01-20T08:59:19.991343+0000","last_peered":"2024-01-20T08:59:19.991343+0000","last_clean":"2024-01-20T08:59:19.991343+0000","last_became_active":"2024-01-20T08:31:10.277316+0000","last_became_peered":"2024-01-20T08:31:10.277316+0000","last_unstale":"2024-01-20T08:59:19.991343+0000","last_undegraded":"2024-01-20T08:59:19.991343+0000","last_fullsized":"2024-01-20T08:59:19.991343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:19.991267+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:19.991267+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.991267+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:36.266081+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"27'136","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.069252+0000","last_change":"2024-01-20T08:59:15.069277+0000","last_active":"2024-01-20T08:59:15.069252+0000","last_peered":"2024-01-20T08:59:15.069252+0000","last_clean":"2024-01-20T08:59:15.069252+0000","last_became_active":"2024-01-20T08:31:16.179324+0000","last_became_peered":"2024-01-20T08:31:16.179324+0000","last_unstale":"2024-01-20T08:59:15.069252+0000","last_undegraded":"2024-01-20T08:59:15.069252+0000","last_fullsized":"2024-01-20T08:59:15.069252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'136","last_scrub_stamp":"2024-01-20T08:59:15.069194+0000","last_deep_scrub":"27'136","last_deep_scrub_stamp":"2024-01-20T08:59:15.069194+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.069194+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:25.628098+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"27'1361","reported_seq":1400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.841575+0000","last_change":"2024-01-20T08:59:11.841614+0000","last_active":"2024-01-20T08:59:11.841575+0000","last_peered":"2024-01-20T08:59:11.841575+0000","last_clean":"2024-01-20T08:59:11.841575+0000","last_became_active":"2024-01-20T08:31:08.624514+0000","last_became_peered":"2024-01-20T08:31:08.624514+0000","last_unstale":"2024-01-20T08:59:11.841575+0000","last_undegraded":"2024-01-20T08:59:11.841575+0000","last_fullsized":"2024-01-20T08:59:11.841575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1361","last_scrub_stamp":"2024-01-20T08:59:11.841510+0000","last_deep_scrub":"27'1361","last_deep_scrub_stamp":"2024-01-20T08:59:11.841510+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.841510+0000","objects_scrubbed":9,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:29:55.568602+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"27'31","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.841361+0000","last_change":"2024-01-20T08:59:09.841382+0000","last_active":"2024-01-20T08:59:09.841361+0000","last_peered":"2024-01-20T08:59:09.841361+0000","last_clean":"2024-01-20T08:59:09.841361+0000","last_became_active":"2024-01-20T08:31:10.280970+0000","last_became_peered":"2024-01-20T08:31:10.280970+0000","last_unstale":"2024-01-20T08:59:09.841361+0000","last_undegraded":"2024-01-20T08:59:09.841361+0000","last_fullsized":"2024-01-20T08:59:09.841361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:09.841311+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:09.841311+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.841311+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:03:42.035846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"27'128","reported_seq":163,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.173162+0000","last_change":"2024-01-20T08:59:19.173186+0000","last_active":"2024-01-20T08:59:19.173162+0000","last_peered":"2024-01-20T08:59:19.173162+0000","last_clean":"2024-01-20T08:59:19.173162+0000","last_became_active":"2024-01-20T08:31:16.182980+0000","last_became_peered":"2024-01-20T08:31:16.182980+0000","last_unstale":"2024-01-20T08:59:19.173162+0000","last_undegraded":"2024-01-20T08:59:19.173162+0000","last_fullsized":"2024-01-20T08:59:19.173162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'128","last_scrub_stamp":"2024-01-20T08:59:19.173096+0000","last_deep_scrub":"27'128","last_deep_scrub_stamp":"2024-01-20T08:59:19.173096+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.173096+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:02:49.049464+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'11","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.770188+0000","last_change":"2024-01-20T08:59:09.770234+0000","last_active":"2024-01-20T08:59:09.770188+0000","last_peered":"2024-01-20T08:59:09.770188+0000","last_clean":"2024-01-20T08:59:09.770188+0000","last_became_active":"2024-01-20T08:31:08.624452+0000","last_became_peered":"2024-01-20T08:31:08.624452+0000","last_unstale":"2024-01-20T08:59:09.770188+0000","last_undegraded":"2024-01-20T08:59:09.770188+0000","last_fullsized":"2024-01-20T08:59:09.770188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-01-20T08:59:09.770124+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-01-20T08:59:09.770124+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.770124+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:39.811326+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"27'47","reported_seq":85,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.475087+0000","last_change":"2024-01-20T08:59:08.475109+0000","last_active":"2024-01-20T08:59:08.475087+0000","last_peered":"2024-01-20T08:59:08.475087+0000","last_clean":"2024-01-20T08:59:08.475087+0000","last_became_active":"2024-01-20T08:31:10.236494+0000","last_became_peered":"2024-01-20T08:31:10.236494+0000","last_unstale":"2024-01-20T08:59:08.475087+0000","last_undegraded":"2024-01-20T08:59:08.475087+0000","last_fullsized":"2024-01-20T08:59:08.475087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-01-20T08:59:08.475024+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-01-20T08:59:08.475024+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.475024+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:26:07.581884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"27'149","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.144297+0000","last_change":"2024-01-20T08:59:18.144342+0000","last_active":"2024-01-20T08:59:18.144297+0000","last_peered":"2024-01-20T08:59:18.144297+0000","last_clean":"2024-01-20T08:59:18.144297+0000","last_became_active":"2024-01-20T08:31:16.181766+0000","last_became_peered":"2024-01-20T08:31:16.181766+0000","last_unstale":"2024-01-20T08:59:18.144297+0000","last_undegraded":"2024-01-20T08:59:18.144297+0000","last_fullsized":"2024-01-20T08:59:18.144297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'149","last_scrub_stamp":"2024-01-20T08:59:18.144206+0000","last_deep_scrub":"27'149","last_deep_scrub_stamp":"2024-01-20T08:59:18.144206+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.144206+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:49.578407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":7763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'35","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:25.759239+0000","last_change":"2024-01-20T08:59:25.759258+0000","last_active":"2024-01-20T08:59:25.759239+0000","last_peered":"2024-01-20T08:59:25.759239+0000","last_clean":"2024-01-20T08:59:25.759239+0000","last_became_active":"2024-01-20T08:31:10.276985+0000","last_became_peered":"2024-01-20T08:31:10.276985+0000","last_unstale":"2024-01-20T08:59:25.759239+0000","last_undegraded":"2024-01-20T08:59:25.759239+0000","last_fullsized":"2024-01-20T08:59:25.759239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:25.759177+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:25.759177+0000","last_clean_scrub_stamp":"2024-01-20T08:59:25.759177+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:26:20.633472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'596","reported_seq":636,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.539017+0000","last_change":"2024-01-20T08:59:15.539043+0000","last_active":"2024-01-20T08:59:15.539017+0000","last_peered":"2024-01-20T08:59:15.539017+0000","last_clean":"2024-01-20T08:59:15.539017+0000","last_became_active":"2024-01-20T08:31:08.624765+0000","last_became_peered":"2024-01-20T08:31:08.624765+0000","last_unstale":"2024-01-20T08:59:15.539017+0000","last_undegraded":"2024-01-20T08:59:15.539017+0000","last_fullsized":"2024-01-20T08:59:15.539017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'596","last_scrub_stamp":"2024-01-20T08:59:15.538958+0000","last_deep_scrub":"27'596","last_deep_scrub_stamp":"2024-01-20T08:59:15.538958+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.538958+0000","objects_scrubbed":3,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:45:35.288982+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":4387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'109","reported_seq":145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:28.847038+0000","last_change":"2024-01-20T08:59:28.847078+0000","last_active":"2024-01-20T08:59:28.847038+0000","last_peered":"2024-01-20T08:59:28.847038+0000","last_clean":"2024-01-20T08:59:28.847038+0000","last_became_active":"2024-01-20T08:31:16.174415+0000","last_became_peered":"2024-01-20T08:31:16.174415+0000","last_unstale":"2024-01-20T08:59:28.847038+0000","last_undegraded":"2024-01-20T08:59:28.847038+0000","last_fullsized":"2024-01-20T08:59:28.847038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'109","last_scrub_stamp":"2024-01-20T08:59:28.846943+0000","last_deep_scrub":"27'109","last_deep_scrub_stamp":"2024-01-20T08:59:28.846943+0000","last_clean_scrub_stamp":"2024-01-20T08:59:28.846943+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:39:37.780732+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"27'35","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.216375+0000","last_change":"2024-01-20T08:59:15.216397+0000","last_active":"2024-01-20T08:59:15.216375+0000","last_peered":"2024-01-20T08:59:15.216375+0000","last_clean":"2024-01-20T08:59:15.216375+0000","last_became_active":"2024-01-20T08:31:10.288774+0000","last_became_peered":"2024-01-20T08:31:10.288774+0000","last_unstale":"2024-01-20T08:59:15.216375+0000","last_undegraded":"2024-01-20T08:59:15.216375+0000","last_fullsized":"2024-01-20T08:59:15.216375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:15.216304+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:15.216304+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.216304+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:38:51.318528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"27'8","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.372586+0000","last_change":"2024-01-20T08:59:11.372586+0000","last_active":"2024-01-20T08:59:11.372586+0000","last_peered":"2024-01-20T08:59:11.372586+0000","last_clean":"2024-01-20T08:59:11.372586+0000","last_became_active":"2024-01-20T08:31:08.594513+0000","last_became_peered":"2024-01-20T08:31:08.594513+0000","last_unstale":"2024-01-20T08:59:11.372586+0000","last_undegraded":"2024-01-20T08:59:11.372586+0000","last_fullsized":"2024-01-20T08:59:11.372586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:11.372441+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:11.372441+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.372441+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:58:44.179767+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"27'149","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.728023+0000","last_change":"2024-01-20T08:59:20.728046+0000","last_active":"2024-01-20T08:59:20.728023+0000","last_peered":"2024-01-20T08:59:20.728023+0000","last_clean":"2024-01-20T08:59:20.728023+0000","last_became_active":"2024-01-20T08:31:16.177597+0000","last_became_peered":"2024-01-20T08:31:16.177597+0000","last_unstale":"2024-01-20T08:59:20.728023+0000","last_undegraded":"2024-01-20T08:59:20.728023+0000","last_fullsized":"2024-01-20T08:59:20.728023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'149","last_scrub_stamp":"2024-01-20T08:59:20.727963+0000","last_deep_scrub":"27'149","last_deep_scrub_stamp":"2024-01-20T08:59:20.727963+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.727963+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:18:29.451689+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'34","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.697472+0000","last_change":"2024-01-20T08:59:07.697472+0000","last_active":"2024-01-20T08:59:07.697472+0000","last_peered":"2024-01-20T08:59:07.697472+0000","last_clean":"2024-01-20T08:59:07.697472+0000","last_became_active":"2024-01-20T08:31:10.235549+0000","last_became_peered":"2024-01-20T08:31:10.235549+0000","last_unstale":"2024-01-20T08:59:07.697472+0000","last_undegraded":"2024-01-20T08:59:07.697472+0000","last_fullsized":"2024-01-20T08:59:07.697472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:07.697392+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:07.697392+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.697392+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:46:28.055701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'17","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.680417+0000","last_change":"2024-01-20T08:59:09.680417+0000","last_active":"2024-01-20T08:59:09.680417+0000","last_peered":"2024-01-20T08:59:09.680417+0000","last_clean":"2024-01-20T08:59:09.680417+0000","last_became_active":"2024-01-20T08:31:08.592380+0000","last_became_peered":"2024-01-20T08:31:08.592380+0000","last_unstale":"2024-01-20T08:59:09.680417+0000","last_undegraded":"2024-01-20T08:59:09.680417+0000","last_fullsized":"2024-01-20T08:59:09.680417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-01-20T08:59:09.680291+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-01-20T08:59:09.680291+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.680291+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:40:27.031404+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"27'121","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.195601+0000","last_change":"2024-01-20T08:59:16.195626+0000","last_active":"2024-01-20T08:59:16.195601+0000","last_peered":"2024-01-20T08:59:16.195601+0000","last_clean":"2024-01-20T08:59:16.195601+0000","last_became_active":"2024-01-20T08:31:16.173334+0000","last_became_peered":"2024-01-20T08:31:16.173334+0000","last_unstale":"2024-01-20T08:59:16.195601+0000","last_undegraded":"2024-01-20T08:59:16.195601+0000","last_fullsized":"2024-01-20T08:59:16.195601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'121","last_scrub_stamp":"2024-01-20T08:59:16.195536+0000","last_deep_scrub":"27'121","last_deep_scrub_stamp":"2024-01-20T08:59:16.195536+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.195536+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:24:51.756954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'32","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.526131+0000","last_change":"2024-01-20T08:59:12.526131+0000","last_active":"2024-01-20T08:59:12.526131+0000","last_peered":"2024-01-20T08:59:12.526131+0000","last_clean":"2024-01-20T08:59:12.526131+0000","last_became_active":"2024-01-20T08:31:10.237102+0000","last_became_peered":"2024-01-20T08:31:10.237102+0000","last_unstale":"2024-01-20T08:59:12.526131+0000","last_undegraded":"2024-01-20T08:59:12.526131+0000","last_fullsized":"2024-01-20T08:59:12.526131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'32","last_scrub_stamp":"2024-01-20T08:59:12.526042+0000","last_deep_scrub":"27'32","last_deep_scrub_stamp":"2024-01-20T08:59:12.526042+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.526042+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:07:13.378040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"27'8","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.785080+0000","last_change":"2024-01-20T08:59:08.785080+0000","last_active":"2024-01-20T08:59:08.785080+0000","last_peered":"2024-01-20T08:59:08.785080+0000","last_clean":"2024-01-20T08:59:08.785080+0000","last_became_active":"2024-01-20T08:31:08.623827+0000","last_became_peered":"2024-01-20T08:31:08.623827+0000","last_unstale":"2024-01-20T08:59:08.785080+0000","last_undegraded":"2024-01-20T08:59:08.785080+0000","last_fullsized":"2024-01-20T08:59:08.785080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:08.784998+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:08.784998+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.784998+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:51.057586+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'112","reported_seq":148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.191749+0000","last_change":"2024-01-20T08:59:20.191788+0000","last_active":"2024-01-20T08:59:20.191749+0000","last_peered":"2024-01-20T08:59:20.191749+0000","last_clean":"2024-01-20T08:59:20.191749+0000","last_became_active":"2024-01-20T08:31:16.183646+0000","last_became_peered":"2024-01-20T08:31:16.183646+0000","last_unstale":"2024-01-20T08:59:20.191749+0000","last_undegraded":"2024-01-20T08:59:20.191749+0000","last_fullsized":"2024-01-20T08:59:20.191749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'112","last_scrub_stamp":"2024-01-20T08:59:20.191680+0000","last_deep_scrub":"27'112","last_deep_scrub_stamp":"2024-01-20T08:59:20.191680+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.191680+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:43:20.989203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'41","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.734360+0000","last_change":"2024-01-20T08:59:16.734360+0000","last_active":"2024-01-20T08:59:16.734360+0000","last_peered":"2024-01-20T08:59:16.734360+0000","last_clean":"2024-01-20T08:59:16.734360+0000","last_became_active":"2024-01-20T08:31:10.235587+0000","last_became_peered":"2024-01-20T08:31:10.235587+0000","last_unstale":"2024-01-20T08:59:16.734360+0000","last_undegraded":"2024-01-20T08:59:16.734360+0000","last_fullsized":"2024-01-20T08:59:16.734360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:16.734239+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:16.734239+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.734239+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:04:37.910402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.601709+0000","last_change":"2024-01-20T08:59:18.601709+0000","last_active":"2024-01-20T08:59:18.601709+0000","last_peered":"2024-01-20T08:59:18.601709+0000","last_clean":"2024-01-20T08:59:18.601709+0000","last_became_active":"2024-01-20T08:31:08.584436+0000","last_became_peered":"2024-01-20T08:31:08.584436+0000","last_unstale":"2024-01-20T08:59:18.601709+0000","last_undegraded":"2024-01-20T08:59:18.601709+0000","last_fullsized":"2024-01-20T08:59:18.601709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-01-20T08:59:18.601614+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-01-20T08:59:18.601614+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.601614+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:32:05.733388+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"27'148","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.122528+0000","last_change":"2024-01-20T08:59:17.122572+0000","last_active":"2024-01-20T08:59:17.122528+0000","last_peered":"2024-01-20T08:59:17.122528+0000","last_clean":"2024-01-20T08:59:17.122528+0000","last_became_active":"2024-01-20T08:31:16.180279+0000","last_became_peered":"2024-01-20T08:31:16.180279+0000","last_unstale":"2024-01-20T08:59:17.122528+0000","last_undegraded":"2024-01-20T08:59:17.122528+0000","last_fullsized":"2024-01-20T08:59:17.122528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'148","last_scrub_stamp":"2024-01-20T08:59:17.122452+0000","last_deep_scrub":"27'148","last_deep_scrub_stamp":"2024-01-20T08:59:17.122452+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.122452+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:55:19.000693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'36","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.699753+0000","last_change":"2024-01-20T08:59:18.699753+0000","last_active":"2024-01-20T08:59:18.699753+0000","last_peered":"2024-01-20T08:59:18.699753+0000","last_clean":"2024-01-20T08:59:18.699753+0000","last_became_active":"2024-01-20T08:31:10.287166+0000","last_became_peered":"2024-01-20T08:31:10.287166+0000","last_unstale":"2024-01-20T08:59:18.699753+0000","last_undegraded":"2024-01-20T08:59:18.699753+0000","last_fullsized":"2024-01-20T08:59:18.699753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:18.699669+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:18.699669+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.699669+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:59:54.426206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'318","reported_seq":359,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.680992+0000","last_change":"2024-01-20T08:59:10.680992+0000","last_active":"2024-01-20T08:59:10.680992+0000","last_peered":"2024-01-20T08:59:10.680992+0000","last_clean":"2024-01-20T08:59:10.680992+0000","last_became_active":"2024-01-20T08:31:08.591827+0000","last_became_peered":"2024-01-20T08:31:08.591827+0000","last_unstale":"2024-01-20T08:59:10.680992+0000","last_undegraded":"2024-01-20T08:59:10.680992+0000","last_fullsized":"2024-01-20T08:59:10.680992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'318","last_scrub_stamp":"2024-01-20T08:59:10.680900+0000","last_deep_scrub":"27'318","last_deep_scrub_stamp":"2024-01-20T08:59:10.680900+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.680900+0000","objects_scrubbed":4,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:02:01.093433+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"27'148","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.715552+0000","last_change":"2024-01-20T08:59:23.715574+0000","last_active":"2024-01-20T08:59:23.715552+0000","last_peered":"2024-01-20T08:59:23.715552+0000","last_clean":"2024-01-20T08:59:23.715552+0000","last_became_active":"2024-01-20T08:31:16.178365+0000","last_became_peered":"2024-01-20T08:31:16.178365+0000","last_unstale":"2024-01-20T08:59:23.715552+0000","last_undegraded":"2024-01-20T08:59:23.715552+0000","last_fullsized":"2024-01-20T08:59:23.715552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'148","last_scrub_stamp":"2024-01-20T08:59:23.715478+0000","last_deep_scrub":"27'148","last_deep_scrub_stamp":"2024-01-20T08:59:23.715478+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.715478+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:03:45.517044+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":11573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"27'35","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.353399+0000","last_change":"2024-01-20T08:59:15.353441+0000","last_active":"2024-01-20T08:59:15.353399+0000","last_peered":"2024-01-20T08:59:15.353399+0000","last_clean":"2024-01-20T08:59:15.353399+0000","last_became_active":"2024-01-20T08:31:10.287133+0000","last_became_peered":"2024-01-20T08:31:10.287133+0000","last_unstale":"2024-01-20T08:59:15.353399+0000","last_undegraded":"2024-01-20T08:59:15.353399+0000","last_fullsized":"2024-01-20T08:59:15.353399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:15.353322+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:15.353322+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.353322+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:45.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"27'11","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.627602+0000","last_change":"2024-01-20T08:59:20.627628+0000","last_active":"2024-01-20T08:59:20.627602+0000","last_peered":"2024-01-20T08:59:20.627602+0000","last_clean":"2024-01-20T08:59:20.627602+0000","last_became_active":"2024-01-20T08:31:08.592152+0000","last_became_peered":"2024-01-20T08:31:08.592152+0000","last_unstale":"2024-01-20T08:59:20.627602+0000","last_undegraded":"2024-01-20T08:59:20.627602+0000","last_fullsized":"2024-01-20T08:59:20.627602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-01-20T08:59:20.627551+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-01-20T08:59:20.627551+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.627551+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:30:21.993177+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"27'126","reported_seq":163,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.293095+0000","last_change":"2024-01-20T08:59:22.293118+0000","last_active":"2024-01-20T08:59:22.293095+0000","last_peered":"2024-01-20T08:59:22.293095+0000","last_clean":"2024-01-20T08:59:22.293095+0000","last_became_active":"2024-01-20T08:31:16.176180+0000","last_became_peered":"2024-01-20T08:31:16.176180+0000","last_unstale":"2024-01-20T08:59:22.293095+0000","last_undegraded":"2024-01-20T08:59:22.293095+0000","last_fullsized":"2024-01-20T08:59:22.293095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'126","last_scrub_stamp":"2024-01-20T08:59:22.293023+0000","last_deep_scrub":"27'126","last_deep_scrub_stamp":"2024-01-20T08:59:22.293023+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.293023+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:52.565087+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'33","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.069746+0000","last_change":"2024-01-20T08:59:13.069774+0000","last_active":"2024-01-20T08:59:13.069746+0000","last_peered":"2024-01-20T08:59:13.069746+0000","last_clean":"2024-01-20T08:59:13.069746+0000","last_became_active":"2024-01-20T08:31:10.287085+0000","last_became_peered":"2024-01-20T08:31:10.287085+0000","last_unstale":"2024-01-20T08:59:13.069746+0000","last_undegraded":"2024-01-20T08:59:13.069746+0000","last_fullsized":"2024-01-20T08:59:13.069746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:13.069682+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:13.069682+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.069682+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:55.342270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.986888+0000","last_change":"2024-01-20T08:59:11.986951+0000","last_active":"2024-01-20T08:59:11.986888+0000","last_peered":"2024-01-20T08:59:11.986888+0000","last_clean":"2024-01-20T08:59:11.986888+0000","last_became_active":"2024-01-20T08:31:08.594015+0000","last_became_peered":"2024-01-20T08:31:08.594015+0000","last_unstale":"2024-01-20T08:59:11.986888+0000","last_undegraded":"2024-01-20T08:59:11.986888+0000","last_fullsized":"2024-01-20T08:59:11.986888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-01-20T08:59:11.986774+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-01-20T08:59:11.986774+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.986774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:42.027975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"27'181","reported_seq":218,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.342183+0000","last_change":"2024-01-20T08:59:21.342206+0000","last_active":"2024-01-20T08:59:21.342183+0000","last_peered":"2024-01-20T08:59:21.342183+0000","last_clean":"2024-01-20T08:59:21.342183+0000","last_became_active":"2024-01-20T08:31:16.160945+0000","last_became_peered":"2024-01-20T08:31:16.160945+0000","last_unstale":"2024-01-20T08:59:21.342183+0000","last_undegraded":"2024-01-20T08:59:21.342183+0000","last_fullsized":"2024-01-20T08:59:21.342183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'181","last_scrub_stamp":"2024-01-20T08:59:21.342122+0000","last_deep_scrub":"27'181","last_deep_scrub_stamp":"2024-01-20T08:59:21.342122+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.342122+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:33:15.086065+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"27'33","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.773565+0000","last_change":"2024-01-20T08:59:13.773603+0000","last_active":"2024-01-20T08:59:13.773565+0000","last_peered":"2024-01-20T08:59:13.773565+0000","last_clean":"2024-01-20T08:59:13.773565+0000","last_became_active":"2024-01-20T08:31:10.195586+0000","last_became_peered":"2024-01-20T08:31:10.195586+0000","last_unstale":"2024-01-20T08:59:13.773565+0000","last_undegraded":"2024-01-20T08:59:13.773565+0000","last_fullsized":"2024-01-20T08:59:13.773565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:13.773482+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:13.773482+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.773482+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:38:42.376489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"27'15","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.624391+0000","last_change":"2024-01-20T08:59:19.624413+0000","last_active":"2024-01-20T08:59:19.624391+0000","last_peered":"2024-01-20T08:59:19.624391+0000","last_clean":"2024-01-20T08:59:19.624391+0000","last_became_active":"2024-01-20T08:31:08.584650+0000","last_became_peered":"2024-01-20T08:31:08.584650+0000","last_unstale":"2024-01-20T08:59:19.624391+0000","last_undegraded":"2024-01-20T08:59:19.624391+0000","last_fullsized":"2024-01-20T08:59:19.624391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-01-20T08:59:19.624338+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-01-20T08:59:19.624338+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.624338+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:26:26.722625+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'113","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.305447+0000","last_change":"2024-01-20T08:59:20.305469+0000","last_active":"2024-01-20T08:59:20.305447+0000","last_peered":"2024-01-20T08:59:20.305447+0000","last_clean":"2024-01-20T08:59:20.305447+0000","last_became_active":"2024-01-20T08:31:16.156854+0000","last_became_peered":"2024-01-20T08:31:16.156854+0000","last_unstale":"2024-01-20T08:59:20.305447+0000","last_undegraded":"2024-01-20T08:59:20.305447+0000","last_fullsized":"2024-01-20T08:59:20.305447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'113","last_scrub_stamp":"2024-01-20T08:59:20.305377+0000","last_deep_scrub":"27'113","last_deep_scrub_stamp":"2024-01-20T08:59:20.305377+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.305377+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:59:56.031913+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"27'32","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.074739+0000","last_change":"2024-01-20T08:59:08.074739+0000","last_active":"2024-01-20T08:59:08.074739+0000","last_peered":"2024-01-20T08:59:08.074739+0000","last_clean":"2024-01-20T08:59:08.074739+0000","last_became_active":"2024-01-20T08:31:10.262838+0000","last_became_peered":"2024-01-20T08:31:10.262838+0000","last_unstale":"2024-01-20T08:59:08.074739+0000","last_undegraded":"2024-01-20T08:59:08.074739+0000","last_fullsized":"2024-01-20T08:59:08.074739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'32","last_scrub_stamp":"2024-01-20T08:59:08.074572+0000","last_deep_scrub":"27'32","last_deep_scrub_stamp":"2024-01-20T08:59:08.074572+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.074572+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:37:10.110970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"27'980","reported_seq":1988,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.303347+0000","last_change":"2024-01-20T08:59:12.303372+0000","last_active":"2024-01-20T08:59:12.303347+0000","last_peered":"2024-01-20T08:59:12.303347+0000","last_clean":"2024-01-20T08:59:12.303347+0000","last_became_active":"2024-01-20T08:31:08.594193+0000","last_became_peered":"2024-01-20T08:31:08.594193+0000","last_unstale":"2024-01-20T08:59:12.303347+0000","last_undegraded":"2024-01-20T08:59:12.303347+0000","last_fullsized":"2024-01-20T08:59:12.303347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'980","last_scrub_stamp":"2024-01-20T08:59:12.303280+0000","last_deep_scrub":"27'980","last_deep_scrub_stamp":"2024-01-20T08:59:12.303280+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.303280+0000","objects_scrubbed":4,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:43:14.836075+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210232,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":976,"num_write_kb":1034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"27'135","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.341955+0000","last_change":"2024-01-20T08:59:23.341955+0000","last_active":"2024-01-20T08:59:23.341955+0000","last_peered":"2024-01-20T08:59:23.341955+0000","last_clean":"2024-01-20T08:59:23.341955+0000","last_became_active":"2024-01-20T08:31:16.177682+0000","last_became_peered":"2024-01-20T08:31:16.177682+0000","last_unstale":"2024-01-20T08:59:23.341955+0000","last_undegraded":"2024-01-20T08:59:23.341955+0000","last_fullsized":"2024-01-20T08:59:23.341955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'135","last_scrub_stamp":"2024-01-20T08:59:23.341833+0000","last_deep_scrub":"27'135","last_deep_scrub_stamp":"2024-01-20T08:59:23.341833+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.341833+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:39.616646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118,"num_write_kb":10261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"27'28","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.373546+0000","last_change":"2024-01-20T08:59:17.373581+0000","last_active":"2024-01-20T08:59:17.373546+0000","last_peered":"2024-01-20T08:59:17.373546+0000","last_clean":"2024-01-20T08:59:17.373546+0000","last_became_active":"2024-01-20T08:31:10.287009+0000","last_became_peered":"2024-01-20T08:31:10.287009+0000","last_unstale":"2024-01-20T08:59:17.373546+0000","last_undegraded":"2024-01-20T08:59:17.373546+0000","last_fullsized":"2024-01-20T08:59:17.373546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-01-20T08:59:17.373441+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-01-20T08:59:17.373441+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.373441+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:30:01.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"27'431","reported_seq":473,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.668712+0000","last_change":"2024-01-20T08:59:21.668735+0000","last_active":"2024-01-20T08:59:21.668712+0000","last_peered":"2024-01-20T08:59:21.668712+0000","last_clean":"2024-01-20T08:59:21.668712+0000","last_became_active":"2024-01-20T08:31:08.595821+0000","last_became_peered":"2024-01-20T08:31:08.595821+0000","last_unstale":"2024-01-20T08:59:21.668712+0000","last_undegraded":"2024-01-20T08:59:21.668712+0000","last_fullsized":"2024-01-20T08:59:21.668712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'431","last_scrub_stamp":"2024-01-20T08:59:21.668659+0000","last_deep_scrub":"27'431","last_deep_scrub_stamp":"2024-01-20T08:59:21.668659+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.668659+0000","objects_scrubbed":2,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:57:22.141144+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":4309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"27'129","reported_seq":166,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:24.051094+0000","last_change":"2024-01-20T08:59:24.051094+0000","last_active":"2024-01-20T08:59:24.051094+0000","last_peered":"2024-01-20T08:59:24.051094+0000","last_clean":"2024-01-20T08:59:24.051094+0000","last_became_active":"2024-01-20T08:31:16.173831+0000","last_became_peered":"2024-01-20T08:31:16.173831+0000","last_unstale":"2024-01-20T08:59:24.051094+0000","last_undegraded":"2024-01-20T08:59:24.051094+0000","last_fullsized":"2024-01-20T08:59:24.051094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'129","last_scrub_stamp":"2024-01-20T08:59:24.050992+0000","last_deep_scrub":"27'129","last_deep_scrub_stamp":"2024-01-20T08:59:24.050992+0000","last_clean_scrub_stamp":"2024-01-20T08:59:24.050992+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:48:10.540045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'32","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.693053+0000","last_change":"2024-01-20T08:59:14.693089+0000","last_active":"2024-01-20T08:59:14.693053+0000","last_peered":"2024-01-20T08:59:14.693053+0000","last_clean":"2024-01-20T08:59:14.693053+0000","last_became_active":"2024-01-20T08:31:10.191130+0000","last_became_peered":"2024-01-20T08:31:10.191130+0000","last_unstale":"2024-01-20T08:59:14.693053+0000","last_undegraded":"2024-01-20T08:59:14.693053+0000","last_fullsized":"2024-01-20T08:59:14.693053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'32","last_scrub_stamp":"2024-01-20T08:59:14.692967+0000","last_deep_scrub":"27'32","last_deep_scrub_stamp":"2024-01-20T08:59:14.692967+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.692967+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:13:16.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.675957+0000","last_change":"2024-01-20T08:59:23.676000+0000","last_active":"2024-01-20T08:59:23.675957+0000","last_peered":"2024-01-20T08:59:23.675957+0000","last_clean":"2024-01-20T08:59:23.675957+0000","last_became_active":"2024-01-20T08:31:08.624454+0000","last_became_peered":"2024-01-20T08:31:08.624454+0000","last_unstale":"2024-01-20T08:59:23.675957+0000","last_undegraded":"2024-01-20T08:59:23.675957+0000","last_fullsized":"2024-01-20T08:59:23.675957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-01-20T08:59:23.675897+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-01-20T08:59:23.675897+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.675897+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:42:54.610826+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":197,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:05.824186+0000","last_change":"2024-01-20T08:59:05.824213+0000","last_active":"2024-01-20T08:59:05.824186+0000","last_peered":"2024-01-20T08:59:05.824186+0000","last_clean":"2024-01-20T08:59:05.824186+0000","last_became_active":"2024-01-20T08:30:47.160999+0000","last_became_peered":"2024-01-20T08:30:47.160999+0000","last_unstale":"2024-01-20T08:59:05.824186+0000","last_undegraded":"2024-01-20T08:59:05.824186+0000","last_fullsized":"2024-01-20T08:59:05.824186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-01-20T08:59:05.824134+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-01-20T08:59:05.824134+0000","last_clean_scrub_stamp":"2024-01-20T08:59:05.824134+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:29:17.877275+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'156","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:29.817889+0000","last_change":"2024-01-20T08:59:29.817912+0000","last_active":"2024-01-20T08:59:29.817889+0000","last_peered":"2024-01-20T08:59:29.817889+0000","last_clean":"2024-01-20T08:59:29.817889+0000","last_became_active":"2024-01-20T08:31:16.175834+0000","last_became_peered":"2024-01-20T08:31:16.175834+0000","last_unstale":"2024-01-20T08:59:29.817889+0000","last_undegraded":"2024-01-20T08:59:29.817889+0000","last_fullsized":"2024-01-20T08:59:29.817889+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'156","last_scrub_stamp":"2024-01-20T08:59:29.817813+0000","last_deep_scrub":"27'156","last_deep_scrub_stamp":"2024-01-20T08:59:29.817813+0000","last_clean_scrub_stamp":"2024-01-20T08:59:29.817813+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:33:29.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":9711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"27'36","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:26.759230+0000","last_change":"2024-01-20T08:59:26.759275+0000","last_active":"2024-01-20T08:59:26.759230+0000","last_peered":"2024-01-20T08:59:26.759230+0000","last_clean":"2024-01-20T08:59:26.759230+0000","last_became_active":"2024-01-20T08:31:10.277623+0000","last_became_peered":"2024-01-20T08:31:10.277623+0000","last_unstale":"2024-01-20T08:59:26.759230+0000","last_undegraded":"2024-01-20T08:59:26.759230+0000","last_fullsized":"2024-01-20T08:59:26.759230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:26.759165+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:26.759165+0000","last_clean_scrub_stamp":"2024-01-20T08:59:26.759165+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:07:43.517630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"27'10","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.374340+0000","last_change":"2024-01-20T08:59:09.374364+0000","last_active":"2024-01-20T08:59:09.374340+0000","last_peered":"2024-01-20T08:59:09.374340+0000","last_clean":"2024-01-20T08:59:09.374340+0000","last_became_active":"2024-01-20T08:31:08.556963+0000","last_became_peered":"2024-01-20T08:31:08.556963+0000","last_unstale":"2024-01-20T08:59:09.374340+0000","last_undegraded":"2024-01-20T08:59:09.374340+0000","last_fullsized":"2024-01-20T08:59:09.374340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-01-20T08:59:09.374281+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-01-20T08:59:09.374281+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.374281+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:23:20.175940+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"27'130","reported_seq":167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.098305+0000","last_change":"2024-01-20T08:59:16.098305+0000","last_active":"2024-01-20T08:59:16.098305+0000","last_peered":"2024-01-20T08:59:16.098305+0000","last_clean":"2024-01-20T08:59:16.098305+0000","last_became_active":"2024-01-20T08:31:16.179509+0000","last_became_peered":"2024-01-20T08:31:16.179509+0000","last_unstale":"2024-01-20T08:59:16.098305+0000","last_undegraded":"2024-01-20T08:59:16.098305+0000","last_fullsized":"2024-01-20T08:59:16.098305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-01-20T08:59:16.098224+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-01-20T08:59:16.098224+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.098224+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:00:13.385005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":4819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'39","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.775637+0000","last_change":"2024-01-20T08:59:15.775659+0000","last_active":"2024-01-20T08:59:15.775637+0000","last_peered":"2024-01-20T08:59:15.775637+0000","last_clean":"2024-01-20T08:59:15.775637+0000","last_became_active":"2024-01-20T08:31:10.235577+0000","last_became_peered":"2024-01-20T08:31:10.235577+0000","last_unstale":"2024-01-20T08:59:15.775637+0000","last_undegraded":"2024-01-20T08:59:15.775637+0000","last_fullsized":"2024-01-20T08:59:15.775637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:15.775578+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:15.775578+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.775578+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:33:09.519522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"27'7","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.046779+0000","last_change":"2024-01-20T08:59:15.046805+0000","last_active":"2024-01-20T08:59:15.046779+0000","last_peered":"2024-01-20T08:59:15.046779+0000","last_clean":"2024-01-20T08:59:15.046779+0000","last_became_active":"2024-01-20T08:31:08.624094+0000","last_became_peered":"2024-01-20T08:31:08.624094+0000","last_unstale":"2024-01-20T08:59:15.046779+0000","last_undegraded":"2024-01-20T08:59:15.046779+0000","last_fullsized":"2024-01-20T08:59:15.046779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-01-20T08:59:15.046715+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-01-20T08:59:15.046715+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.046715+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:55:16.924956+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"27'188","reported_seq":224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:26.093943+0000","last_change":"2024-01-20T08:59:26.093971+0000","last_active":"2024-01-20T08:59:26.093943+0000","last_peered":"2024-01-20T08:59:26.093943+0000","last_clean":"2024-01-20T08:59:26.093943+0000","last_became_active":"2024-01-20T08:31:16.177441+0000","last_became_peered":"2024-01-20T08:31:16.177441+0000","last_unstale":"2024-01-20T08:59:26.093943+0000","last_undegraded":"2024-01-20T08:59:26.093943+0000","last_fullsized":"2024-01-20T08:59:26.093943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'188","last_scrub_stamp":"2024-01-20T08:59:26.093851+0000","last_deep_scrub":"27'188","last_deep_scrub_stamp":"2024-01-20T08:59:26.093851+0000","last_clean_scrub_stamp":"2024-01-20T08:59:26.093851+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:38:36.846978+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'35","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.150876+0000","last_change":"2024-01-20T08:59:19.150932+0000","last_active":"2024-01-20T08:59:19.150876+0000","last_peered":"2024-01-20T08:59:19.150876+0000","last_clean":"2024-01-20T08:59:19.150876+0000","last_became_active":"2024-01-20T08:31:44.212617+0000","last_became_peered":"2024-01-20T08:31:44.212617+0000","last_unstale":"2024-01-20T08:59:19.150876+0000","last_undegraded":"2024-01-20T08:59:19.150876+0000","last_fullsized":"2024-01-20T08:59:19.150876+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:19.150805+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:19.150805+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.150805+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:39:22.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'10","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.722710+0000","last_change":"2024-01-20T08:59:12.722733+0000","last_active":"2024-01-20T08:59:12.722710+0000","last_peered":"2024-01-20T08:59:12.722710+0000","last_clean":"2024-01-20T08:59:12.722710+0000","last_became_active":"2024-01-20T08:31:08.594150+0000","last_became_peered":"2024-01-20T08:31:08.594150+0000","last_unstale":"2024-01-20T08:59:12.722710+0000","last_undegraded":"2024-01-20T08:59:12.722710+0000","last_fullsized":"2024-01-20T08:59:12.722710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-01-20T08:59:12.722653+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-01-20T08:59:12.722653+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.722653+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:54.995241+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"27'145","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:30.861211+0000","last_change":"2024-01-20T08:59:30.861232+0000","last_active":"2024-01-20T08:59:30.861211+0000","last_peered":"2024-01-20T08:59:30.861211+0000","last_clean":"2024-01-20T08:59:30.861211+0000","last_became_active":"2024-01-20T08:31:16.179383+0000","last_became_peered":"2024-01-20T08:31:16.179383+0000","last_unstale":"2024-01-20T08:59:30.861211+0000","last_undegraded":"2024-01-20T08:59:30.861211+0000","last_fullsized":"2024-01-20T08:59:30.861211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'145","last_scrub_stamp":"2024-01-20T08:59:30.861148+0000","last_deep_scrub":"27'145","last_deep_scrub_stamp":"2024-01-20T08:59:30.861148+0000","last_clean_scrub_stamp":"2024-01-20T08:59:30.861148+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:20:16.543789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'21","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.049300+0000","last_change":"2024-01-20T08:59:09.049341+0000","last_active":"2024-01-20T08:59:09.049300+0000","last_peered":"2024-01-20T08:59:09.049300+0000","last_clean":"2024-01-20T08:59:09.049300+0000","last_became_active":"2024-01-20T08:31:10.194943+0000","last_became_peered":"2024-01-20T08:31:10.194943+0000","last_unstale":"2024-01-20T08:59:09.049300+0000","last_undegraded":"2024-01-20T08:59:09.049300+0000","last_fullsized":"2024-01-20T08:59:09.049300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-01-20T08:59:09.049204+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-01-20T08:59:09.049204+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.049204+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:19.502084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'5","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.740446+0000","last_change":"2024-01-20T08:59:07.740470+0000","last_active":"2024-01-20T08:59:07.740446+0000","last_peered":"2024-01-20T08:59:07.740446+0000","last_clean":"2024-01-20T08:59:07.740446+0000","last_became_active":"2024-01-20T08:31:08.594619+0000","last_became_peered":"2024-01-20T08:31:08.594619+0000","last_unstale":"2024-01-20T08:59:07.740446+0000","last_undegraded":"2024-01-20T08:59:07.740446+0000","last_fullsized":"2024-01-20T08:59:07.740446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-01-20T08:59:07.740394+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-01-20T08:59:07.740394+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.740394+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:21.774310+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"27'160","reported_seq":197,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.877502+0000","last_change":"2024-01-20T08:59:12.877502+0000","last_active":"2024-01-20T08:59:12.877502+0000","last_peered":"2024-01-20T08:59:12.877502+0000","last_clean":"2024-01-20T08:59:12.877502+0000","last_became_active":"2024-01-20T08:31:16.178201+0000","last_became_peered":"2024-01-20T08:31:16.178201+0000","last_unstale":"2024-01-20T08:59:12.877502+0000","last_undegraded":"2024-01-20T08:59:12.877502+0000","last_fullsized":"2024-01-20T08:59:12.877502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'160","last_scrub_stamp":"2024-01-20T08:59:12.877413+0000","last_deep_scrub":"27'160","last_deep_scrub_stamp":"2024-01-20T08:59:12.877413+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.877413+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:23.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":10942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"27'35","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.547738+0000","last_change":"2024-01-20T08:59:13.547770+0000","last_active":"2024-01-20T08:59:13.547738+0000","last_peered":"2024-01-20T08:59:13.547738+0000","last_clean":"2024-01-20T08:59:13.547738+0000","last_became_active":"2024-01-20T08:31:10.235185+0000","last_became_peered":"2024-01-20T08:31:10.235185+0000","last_unstale":"2024-01-20T08:59:13.547738+0000","last_undegraded":"2024-01-20T08:59:13.547738+0000","last_fullsized":"2024-01-20T08:59:13.547738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:13.547656+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:13.547656+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.547656+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:55:30.322108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"27'29","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.679673+0000","last_change":"2024-01-20T08:59:08.679673+0000","last_active":"2024-01-20T08:59:08.679673+0000","last_peered":"2024-01-20T08:59:08.679673+0000","last_clean":"2024-01-20T08:59:08.679673+0000","last_became_active":"2024-01-20T08:31:08.591848+0000","last_became_peered":"2024-01-20T08:31:08.591848+0000","last_unstale":"2024-01-20T08:59:08.679673+0000","last_undegraded":"2024-01-20T08:59:08.679673+0000","last_fullsized":"2024-01-20T08:59:08.679673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-01-20T08:59:08.679555+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-01-20T08:59:08.679555+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.679555+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:58:41.486881+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"27'142","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:25.047670+0000","last_change":"2024-01-20T08:59:25.047717+0000","last_active":"2024-01-20T08:59:25.047670+0000","last_peered":"2024-01-20T08:59:25.047670+0000","last_clean":"2024-01-20T08:59:25.047670+0000","last_became_active":"2024-01-20T08:31:16.173865+0000","last_became_peered":"2024-01-20T08:31:16.173865+0000","last_unstale":"2024-01-20T08:59:25.047670+0000","last_undegraded":"2024-01-20T08:59:25.047670+0000","last_fullsized":"2024-01-20T08:59:25.047670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'142","last_scrub_stamp":"2024-01-20T08:59:25.047568+0000","last_deep_scrub":"27'142","last_deep_scrub_stamp":"2024-01-20T08:59:25.047568+0000","last_clean_scrub_stamp":"2024-01-20T08:59:25.047568+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:35:06.679443+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'29","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.785242+0000","last_change":"2024-01-20T08:59:07.785242+0000","last_active":"2024-01-20T08:59:07.785242+0000","last_peered":"2024-01-20T08:59:07.785242+0000","last_clean":"2024-01-20T08:59:07.785242+0000","last_became_active":"2024-01-20T08:31:10.276792+0000","last_became_peered":"2024-01-20T08:31:10.276792+0000","last_unstale":"2024-01-20T08:59:07.785242+0000","last_undegraded":"2024-01-20T08:59:07.785242+0000","last_fullsized":"2024-01-20T08:59:07.785242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-01-20T08:59:07.785115+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-01-20T08:59:07.785115+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.785115+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:03:31.674517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"27'13","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.790415+0000","last_change":"2024-01-20T08:59:06.790415+0000","last_active":"2024-01-20T08:59:06.790415+0000","last_peered":"2024-01-20T08:59:06.790415+0000","last_clean":"2024-01-20T08:59:06.790415+0000","last_became_active":"2024-01-20T08:31:08.583420+0000","last_became_peered":"2024-01-20T08:31:08.583420+0000","last_unstale":"2024-01-20T08:59:06.790415+0000","last_undegraded":"2024-01-20T08:59:06.790415+0000","last_fullsized":"2024-01-20T08:59:06.790415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-01-20T08:59:06.790291+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-01-20T08:59:06.790291+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.790291+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:57.202825+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1169,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"27'133","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:27.143178+0000","last_change":"2024-01-20T08:59:27.143205+0000","last_active":"2024-01-20T08:59:27.143178+0000","last_peered":"2024-01-20T08:59:27.143178+0000","last_clean":"2024-01-20T08:59:27.143178+0000","last_became_active":"2024-01-20T08:31:16.178421+0000","last_became_peered":"2024-01-20T08:31:16.178421+0000","last_unstale":"2024-01-20T08:59:27.143178+0000","last_undegraded":"2024-01-20T08:59:27.143178+0000","last_fullsized":"2024-01-20T08:59:27.143178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'133","last_scrub_stamp":"2024-01-20T08:59:27.143090+0000","last_deep_scrub":"27'133","last_deep_scrub_stamp":"2024-01-20T08:59:27.143090+0000","last_clean_scrub_stamp":"2024-01-20T08:59:27.143090+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:39:35.182978+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'48","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.483859+0000","last_change":"2024-01-20T08:59:07.483859+0000","last_active":"2024-01-20T08:59:07.483859+0000","last_peered":"2024-01-20T08:59:07.483859+0000","last_clean":"2024-01-20T08:59:07.483859+0000","last_became_active":"2024-01-20T08:31:10.235628+0000","last_became_peered":"2024-01-20T08:31:10.235628+0000","last_unstale":"2024-01-20T08:59:07.483859+0000","last_undegraded":"2024-01-20T08:59:07.483859+0000","last_fullsized":"2024-01-20T08:59:07.483859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-01-20T08:59:07.483752+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-01-20T08:59:07.483752+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.483752+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:55:39.655254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.570873+0000","last_change":"2024-01-20T08:59:16.570914+0000","last_active":"2024-01-20T08:59:16.570873+0000","last_peered":"2024-01-20T08:59:16.570873+0000","last_clean":"2024-01-20T08:59:16.570873+0000","last_became_active":"2024-01-20T08:31:08.583840+0000","last_became_peered":"2024-01-20T08:31:08.583840+0000","last_unstale":"2024-01-20T08:59:16.570873+0000","last_undegraded":"2024-01-20T08:59:16.570873+0000","last_fullsized":"2024-01-20T08:59:16.570873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-01-20T08:59:16.570787+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-01-20T08:59:16.570787+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.570787+0000","objects_scrubbed":3,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:48:17.320248+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"27'116","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.330579+0000","last_change":"2024-01-20T08:59:19.330579+0000","last_active":"2024-01-20T08:59:19.330579+0000","last_peered":"2024-01-20T08:59:19.330579+0000","last_clean":"2024-01-20T08:59:19.330579+0000","last_became_active":"2024-01-20T08:31:16.154727+0000","last_became_peered":"2024-01-20T08:31:16.154727+0000","last_unstale":"2024-01-20T08:59:19.330579+0000","last_undegraded":"2024-01-20T08:59:19.330579+0000","last_fullsized":"2024-01-20T08:59:19.330579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'116","last_scrub_stamp":"2024-01-20T08:59:19.330498+0000","last_deep_scrub":"27'116","last_deep_scrub_stamp":"2024-01-20T08:59:19.330498+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.330498+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:38:47.933506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"27'40","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.393935+0000","last_change":"2024-01-20T08:59:13.393974+0000","last_active":"2024-01-20T08:59:13.393935+0000","last_peered":"2024-01-20T08:59:13.393935+0000","last_clean":"2024-01-20T08:59:13.393935+0000","last_became_active":"2024-01-20T08:31:10.284820+0000","last_became_peered":"2024-01-20T08:31:10.284820+0000","last_unstale":"2024-01-20T08:59:13.393935+0000","last_undegraded":"2024-01-20T08:59:13.393935+0000","last_fullsized":"2024-01-20T08:59:13.393935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'40","last_scrub_stamp":"2024-01-20T08:59:13.393836+0000","last_deep_scrub":"27'40","last_deep_scrub_stamp":"2024-01-20T08:59:13.393836+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.393836+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:27.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"27'117","reported_seq":157,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.293415+0000","last_change":"2024-01-20T08:59:10.293444+0000","last_active":"2024-01-20T08:59:10.293415+0000","last_peered":"2024-01-20T08:59:10.293415+0000","last_clean":"2024-01-20T08:59:10.293415+0000","last_became_active":"2024-01-20T08:31:08.585371+0000","last_became_peered":"2024-01-20T08:31:08.585371+0000","last_unstale":"2024-01-20T08:59:10.293415+0000","last_undegraded":"2024-01-20T08:59:10.293415+0000","last_fullsized":"2024-01-20T08:59:10.293415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'117","last_scrub_stamp":"2024-01-20T08:59:10.293359+0000","last_deep_scrub":"27'117","last_deep_scrub_stamp":"2024-01-20T08:59:10.293359+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.293359+0000","objects_scrubbed":7,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:29:41.075136+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069576,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"27'126","reported_seq":161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:27.803610+0000","last_change":"2024-01-20T08:59:27.803632+0000","last_active":"2024-01-20T08:59:27.803610+0000","last_peered":"2024-01-20T08:59:27.803610+0000","last_clean":"2024-01-20T08:59:27.803610+0000","last_became_active":"2024-01-20T08:31:16.160587+0000","last_became_peered":"2024-01-20T08:31:16.160587+0000","last_unstale":"2024-01-20T08:59:27.803610+0000","last_undegraded":"2024-01-20T08:59:27.803610+0000","last_fullsized":"2024-01-20T08:59:27.803610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'126","last_scrub_stamp":"2024-01-20T08:59:27.803543+0000","last_deep_scrub":"27'126","last_deep_scrub_stamp":"2024-01-20T08:59:27.803543+0000","last_clean_scrub_stamp":"2024-01-20T08:59:27.803543+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:14:59.209609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'36","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.713490+0000","last_change":"2024-01-20T08:59:19.713510+0000","last_active":"2024-01-20T08:59:19.713490+0000","last_peered":"2024-01-20T08:59:19.713490+0000","last_clean":"2024-01-20T08:59:19.713490+0000","last_became_active":"2024-01-20T08:31:10.287675+0000","last_became_peered":"2024-01-20T08:31:10.287675+0000","last_unstale":"2024-01-20T08:59:19.713490+0000","last_undegraded":"2024-01-20T08:59:19.713490+0000","last_fullsized":"2024-01-20T08:59:19.713490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:19.713435+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:19.713435+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.713435+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:26:17.789473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'838","reported_seq":878,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.086932+0000","last_change":"2024-01-20T08:59:06.086932+0000","last_active":"2024-01-20T08:59:06.086932+0000","last_peered":"2024-01-20T08:59:06.086932+0000","last_clean":"2024-01-20T08:59:06.086932+0000","last_became_active":"2024-01-20T08:31:08.624933+0000","last_became_peered":"2024-01-20T08:31:08.624933+0000","last_unstale":"2024-01-20T08:59:06.086932+0000","last_undegraded":"2024-01-20T08:59:06.086932+0000","last_fullsized":"2024-01-20T08:59:06.086932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'838","last_scrub_stamp":"2024-01-20T08:59:06.086846+0000","last_deep_scrub":"27'838","last_deep_scrub_stamp":"2024-01-20T08:59:06.086846+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.086846+0000","objects_scrubbed":6,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:03:38.281381+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"27'135","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.740285+0000","last_change":"2024-01-20T08:59:20.740325+0000","last_active":"2024-01-20T08:59:20.740285+0000","last_peered":"2024-01-20T08:59:20.740285+0000","last_clean":"2024-01-20T08:59:20.740285+0000","last_became_active":"2024-01-20T08:31:16.160992+0000","last_became_peered":"2024-01-20T08:31:16.160992+0000","last_unstale":"2024-01-20T08:59:20.740285+0000","last_undegraded":"2024-01-20T08:59:20.740285+0000","last_fullsized":"2024-01-20T08:59:20.740285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'135","last_scrub_stamp":"2024-01-20T08:59:20.740199+0000","last_deep_scrub":"27'135","last_deep_scrub_stamp":"2024-01-20T08:59:20.740199+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.740199+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:51.012263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"27'31","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.439085+0000","last_change":"2024-01-20T08:59:09.439085+0000","last_active":"2024-01-20T08:59:09.439085+0000","last_peered":"2024-01-20T08:59:09.439085+0000","last_clean":"2024-01-20T08:59:09.439085+0000","last_became_active":"2024-01-20T08:31:10.235390+0000","last_became_peered":"2024-01-20T08:31:10.235390+0000","last_unstale":"2024-01-20T08:59:09.439085+0000","last_undegraded":"2024-01-20T08:59:09.439085+0000","last_fullsized":"2024-01-20T08:59:09.439085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:09.438951+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:09.438951+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.438951+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:13:50.002386+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"27'655","reported_seq":694,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.257018+0000","last_change":"2024-01-20T08:59:11.257062+0000","last_active":"2024-01-20T08:59:11.257018+0000","last_peered":"2024-01-20T08:59:11.257018+0000","last_clean":"2024-01-20T08:59:11.257018+0000","last_became_active":"2024-01-20T08:31:08.591161+0000","last_became_peered":"2024-01-20T08:31:08.591161+0000","last_unstale":"2024-01-20T08:59:11.257018+0000","last_undegraded":"2024-01-20T08:59:11.257018+0000","last_fullsized":"2024-01-20T08:59:11.257018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'655","last_scrub_stamp":"2024-01-20T08:59:11.256922+0000","last_deep_scrub":"27'655","last_deep_scrub_stamp":"2024-01-20T08:59:11.256922+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.256922+0000","objects_scrubbed":4,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:30:33.251237+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"27'155","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.044048+0000","last_change":"2024-01-20T08:59:14.044048+0000","last_active":"2024-01-20T08:59:14.044048+0000","last_peered":"2024-01-20T08:59:14.044048+0000","last_clean":"2024-01-20T08:59:14.044048+0000","last_became_active":"2024-01-20T08:31:16.179290+0000","last_became_peered":"2024-01-20T08:31:16.179290+0000","last_unstale":"2024-01-20T08:59:14.044048+0000","last_undegraded":"2024-01-20T08:59:14.044048+0000","last_fullsized":"2024-01-20T08:59:14.044048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'155","last_scrub_stamp":"2024-01-20T08:59:14.043962+0000","last_deep_scrub":"27'155","last_deep_scrub_stamp":"2024-01-20T08:59:14.043962+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.043962+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:44.085163+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":23255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'39","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:09.059523+0000","last_change":"2024-01-20T08:59:09.059546+0000","last_active":"2024-01-20T08:59:09.059523+0000","last_peered":"2024-01-20T08:59:09.059523+0000","last_clean":"2024-01-20T08:59:09.059523+0000","last_became_active":"2024-01-20T08:31:10.262800+0000","last_became_peered":"2024-01-20T08:31:10.262800+0000","last_unstale":"2024-01-20T08:59:09.059523+0000","last_undegraded":"2024-01-20T08:59:09.059523+0000","last_fullsized":"2024-01-20T08:59:09.059523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:09.059430+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:09.059430+0000","last_clean_scrub_stamp":"2024-01-20T08:59:09.059430+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:01:59.471964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'8","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.792880+0000","last_change":"2024-01-20T08:59:10.792908+0000","last_active":"2024-01-20T08:59:10.792880+0000","last_peered":"2024-01-20T08:59:10.792880+0000","last_clean":"2024-01-20T08:59:10.792880+0000","last_became_active":"2024-01-20T08:31:08.623870+0000","last_became_peered":"2024-01-20T08:31:08.623870+0000","last_unstale":"2024-01-20T08:59:10.792880+0000","last_undegraded":"2024-01-20T08:59:10.792880+0000","last_fullsized":"2024-01-20T08:59:10.792880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:10.792812+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:10.792812+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.792812+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:13:12.769316+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"27'124","reported_seq":160,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.149518+0000","last_change":"2024-01-20T08:59:18.149518+0000","last_active":"2024-01-20T08:59:18.149518+0000","last_peered":"2024-01-20T08:59:18.149518+0000","last_clean":"2024-01-20T08:59:18.149518+0000","last_became_active":"2024-01-20T08:31:16.179180+0000","last_became_peered":"2024-01-20T08:31:16.179180+0000","last_unstale":"2024-01-20T08:59:18.149518+0000","last_undegraded":"2024-01-20T08:59:18.149518+0000","last_fullsized":"2024-01-20T08:59:18.149518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'124","last_scrub_stamp":"2024-01-20T08:59:18.149440+0000","last_deep_scrub":"27'124","last_deep_scrub_stamp":"2024-01-20T08:59:18.149440+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.149440+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:34:05.755461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"27'42","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.078143+0000","last_change":"2024-01-20T08:59:11.078180+0000","last_active":"2024-01-20T08:59:11.078143+0000","last_peered":"2024-01-20T08:59:11.078143+0000","last_clean":"2024-01-20T08:59:11.078143+0000","last_became_active":"2024-01-20T08:31:10.234069+0000","last_became_peered":"2024-01-20T08:31:10.234069+0000","last_unstale":"2024-01-20T08:59:11.078143+0000","last_undegraded":"2024-01-20T08:59:11.078143+0000","last_fullsized":"2024-01-20T08:59:11.078143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:11.078073+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:11.078073+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.078073+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:02:01.490607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'15","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.836288+0000","last_change":"2024-01-20T08:59:06.836330+0000","last_active":"2024-01-20T08:59:06.836288+0000","last_peered":"2024-01-20T08:59:06.836288+0000","last_clean":"2024-01-20T08:59:06.836288+0000","last_became_active":"2024-01-20T08:31:08.625573+0000","last_became_peered":"2024-01-20T08:31:08.625573+0000","last_unstale":"2024-01-20T08:59:06.836288+0000","last_undegraded":"2024-01-20T08:59:06.836288+0000","last_fullsized":"2024-01-20T08:59:06.836288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-01-20T08:59:06.836229+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-01-20T08:59:06.836229+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.836229+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:58:53.564126+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"27'155","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:19.723501+0000","last_change":"2024-01-20T08:59:19.723523+0000","last_active":"2024-01-20T08:59:19.723501+0000","last_peered":"2024-01-20T08:59:19.723501+0000","last_clean":"2024-01-20T08:59:19.723501+0000","last_became_active":"2024-01-20T08:31:16.154815+0000","last_became_peered":"2024-01-20T08:31:16.154815+0000","last_unstale":"2024-01-20T08:59:19.723501+0000","last_undegraded":"2024-01-20T08:59:19.723501+0000","last_fullsized":"2024-01-20T08:59:19.723501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'155","last_scrub_stamp":"2024-01-20T08:59:19.723439+0000","last_deep_scrub":"27'155","last_deep_scrub_stamp":"2024-01-20T08:59:19.723439+0000","last_clean_scrub_stamp":"2024-01-20T08:59:19.723439+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T12:28:39.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"27'130","reported_seq":168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.704643+0000","last_change":"2024-01-20T08:59:22.704685+0000","last_active":"2024-01-20T08:59:22.704643+0000","last_peered":"2024-01-20T08:59:22.704643+0000","last_clean":"2024-01-20T08:59:22.704643+0000","last_became_active":"2024-01-20T08:31:16.174397+0000","last_became_peered":"2024-01-20T08:31:16.174397+0000","last_unstale":"2024-01-20T08:59:22.704643+0000","last_undegraded":"2024-01-20T08:59:22.704643+0000","last_fullsized":"2024-01-20T08:59:22.704643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-01-20T08:59:22.704542+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-01-20T08:59:22.704542+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.704542+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:20:09.614121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":8002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'42","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.692010+0000","last_change":"2024-01-20T08:59:17.692033+0000","last_active":"2024-01-20T08:59:17.692010+0000","last_peered":"2024-01-20T08:59:17.692010+0000","last_clean":"2024-01-20T08:59:17.692010+0000","last_became_active":"2024-01-20T08:31:10.276873+0000","last_became_peered":"2024-01-20T08:31:10.276873+0000","last_unstale":"2024-01-20T08:59:17.692010+0000","last_undegraded":"2024-01-20T08:59:17.692010+0000","last_fullsized":"2024-01-20T08:59:17.692010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:17.691942+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:17.691942+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.691942+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:38:46.294949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"27'8","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.666558+0000","last_change":"2024-01-20T08:59:22.666579+0000","last_active":"2024-01-20T08:59:22.666558+0000","last_peered":"2024-01-20T08:59:22.666558+0000","last_clean":"2024-01-20T08:59:22.666558+0000","last_became_active":"2024-01-20T08:31:08.623868+0000","last_became_peered":"2024-01-20T08:31:08.623868+0000","last_unstale":"2024-01-20T08:59:22.666558+0000","last_undegraded":"2024-01-20T08:59:22.666558+0000","last_fullsized":"2024-01-20T08:59:22.666558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-01-20T08:59:22.666499+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-01-20T08:59:22.666499+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.666499+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:25:52.125761+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"27'143","reported_seq":180,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:31.817919+0000","last_change":"2024-01-20T08:59:31.817949+0000","last_active":"2024-01-20T08:59:31.817919+0000","last_peered":"2024-01-20T08:59:31.817919+0000","last_clean":"2024-01-20T08:59:31.817919+0000","last_became_active":"2024-01-20T08:31:16.179894+0000","last_became_peered":"2024-01-20T08:31:16.179894+0000","last_unstale":"2024-01-20T08:59:31.817919+0000","last_undegraded":"2024-01-20T08:59:31.817919+0000","last_fullsized":"2024-01-20T08:59:31.817919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'143","last_scrub_stamp":"2024-01-20T08:59:31.817856+0000","last_deep_scrub":"27'143","last_deep_scrub_stamp":"2024-01-20T08:59:31.817856+0000","last_clean_scrub_stamp":"2024-01-20T08:59:31.817856+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:24:54.851084+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":18955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'41","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:23.051349+0000","last_change":"2024-01-20T08:59:23.051374+0000","last_active":"2024-01-20T08:59:23.051349+0000","last_peered":"2024-01-20T08:59:23.051349+0000","last_clean":"2024-01-20T08:59:23.051349+0000","last_became_active":"2024-01-20T08:31:10.287062+0000","last_became_peered":"2024-01-20T08:31:10.287062+0000","last_unstale":"2024-01-20T08:59:23.051349+0000","last_undegraded":"2024-01-20T08:59:23.051349+0000","last_fullsized":"2024-01-20T08:59:23.051349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:23.051275+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:23.051275+0000","last_clean_scrub_stamp":"2024-01-20T08:59:23.051275+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:07:58.886758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"27'9","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.345284+0000","last_change":"2024-01-20T08:59:10.345326+0000","last_active":"2024-01-20T08:59:10.345284+0000","last_peered":"2024-01-20T08:59:10.345284+0000","last_clean":"2024-01-20T08:59:10.345284+0000","last_became_active":"2024-01-20T08:31:08.556893+0000","last_became_peered":"2024-01-20T08:31:08.556893+0000","last_unstale":"2024-01-20T08:59:10.345284+0000","last_undegraded":"2024-01-20T08:59:10.345284+0000","last_fullsized":"2024-01-20T08:59:10.345284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-01-20T08:59:10.345200+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-01-20T08:59:10.345200+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.345200+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:49:06.776395+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"27'679","reported_seq":718,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.614378+0000","last_change":"2024-01-20T08:59:17.614404+0000","last_active":"2024-01-20T08:59:17.614378+0000","last_peered":"2024-01-20T08:59:17.614378+0000","last_clean":"2024-01-20T08:59:17.614378+0000","last_became_active":"2024-01-20T08:31:08.584380+0000","last_became_peered":"2024-01-20T08:31:08.584380+0000","last_unstale":"2024-01-20T08:59:17.614378+0000","last_undegraded":"2024-01-20T08:59:17.614378+0000","last_fullsized":"2024-01-20T08:59:17.614378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'679","last_scrub_stamp":"2024-01-20T08:59:17.614324+0000","last_deep_scrub":"27'679","last_deep_scrub_stamp":"2024-01-20T08:59:17.614324+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.614324+0000","objects_scrubbed":4,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:33:59.859639+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'39","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:15.742148+0000","last_change":"2024-01-20T08:59:15.742170+0000","last_active":"2024-01-20T08:59:15.742148+0000","last_peered":"2024-01-20T08:59:15.742148+0000","last_clean":"2024-01-20T08:59:15.742148+0000","last_became_active":"2024-01-20T08:31:10.195020+0000","last_became_peered":"2024-01-20T08:31:10.195020+0000","last_unstale":"2024-01-20T08:59:15.742148+0000","last_undegraded":"2024-01-20T08:59:15.742148+0000","last_fullsized":"2024-01-20T08:59:15.742148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:15.742087+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:15.742087+0000","last_clean_scrub_stamp":"2024-01-20T08:59:15.742087+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:29:59.469179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"27'89","reported_seq":124,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:21.735493+0000","last_change":"2024-01-20T08:59:21.735538+0000","last_active":"2024-01-20T08:59:21.735493+0000","last_peered":"2024-01-20T08:59:21.735493+0000","last_clean":"2024-01-20T08:59:21.735493+0000","last_became_active":"2024-01-20T08:31:16.161579+0000","last_became_peered":"2024-01-20T08:31:16.161579+0000","last_unstale":"2024-01-20T08:59:21.735493+0000","last_undegraded":"2024-01-20T08:59:21.735493+0000","last_fullsized":"2024-01-20T08:59:21.735493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'89","last_scrub_stamp":"2024-01-20T08:59:21.735407+0000","last_deep_scrub":"27'89","last_deep_scrub_stamp":"2024-01-20T08:59:21.735407+0000","last_clean_scrub_stamp":"2024-01-20T08:59:21.735407+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:38.010221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":9333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'691","reported_seq":730,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.000879+0000","last_change":"2024-01-20T08:59:13.000926+0000","last_active":"2024-01-20T08:59:13.000879+0000","last_peered":"2024-01-20T08:59:13.000879+0000","last_clean":"2024-01-20T08:59:13.000879+0000","last_became_active":"2024-01-20T08:31:08.595656+0000","last_became_peered":"2024-01-20T08:31:08.595656+0000","last_unstale":"2024-01-20T08:59:13.000879+0000","last_undegraded":"2024-01-20T08:59:13.000879+0000","last_fullsized":"2024-01-20T08:59:13.000879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'691","last_scrub_stamp":"2024-01-20T08:59:13.000787+0000","last_deep_scrub":"27'691","last_deep_scrub_stamp":"2024-01-20T08:59:13.000787+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.000787+0000","objects_scrubbed":3,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:00:23.559691+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"27'33","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.805565+0000","last_change":"2024-01-20T08:59:08.805605+0000","last_active":"2024-01-20T08:59:08.805565+0000","last_peered":"2024-01-20T08:59:08.805565+0000","last_clean":"2024-01-20T08:59:08.805565+0000","last_became_active":"2024-01-20T08:31:10.278224+0000","last_became_peered":"2024-01-20T08:31:10.278224+0000","last_unstale":"2024-01-20T08:59:08.805565+0000","last_undegraded":"2024-01-20T08:59:08.805565+0000","last_fullsized":"2024-01-20T08:59:08.805565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:08.805473+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:08.805473+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.805473+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T16:42:35.530260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"27'147","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.146924+0000","last_change":"2024-01-20T08:59:17.146946+0000","last_active":"2024-01-20T08:59:17.146924+0000","last_peered":"2024-01-20T08:59:17.146924+0000","last_clean":"2024-01-20T08:59:17.146924+0000","last_became_active":"2024-01-20T08:31:16.179115+0000","last_became_peered":"2024-01-20T08:31:16.179115+0000","last_unstale":"2024-01-20T08:59:17.146924+0000","last_undegraded":"2024-01-20T08:59:17.146924+0000","last_fullsized":"2024-01-20T08:59:17.146924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'147","last_scrub_stamp":"2024-01-20T08:59:17.146851+0000","last_deep_scrub":"27'147","last_deep_scrub_stamp":"2024-01-20T08:59:17.146851+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.146851+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:50:24.411608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":10430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"27'50","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.087872+0000","last_change":"2024-01-20T08:59:12.087893+0000","last_active":"2024-01-20T08:59:12.087872+0000","last_peered":"2024-01-20T08:59:12.087872+0000","last_clean":"2024-01-20T08:59:12.087872+0000","last_became_active":"2024-01-20T08:31:10.287074+0000","last_became_peered":"2024-01-20T08:31:10.287074+0000","last_unstale":"2024-01-20T08:59:12.087872+0000","last_undegraded":"2024-01-20T08:59:12.087872+0000","last_fullsized":"2024-01-20T08:59:12.087872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-01-20T08:59:12.087798+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-01-20T08:59:12.087798+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.087798+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:26.121715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"27'50","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.252254+0000","last_change":"2024-01-20T08:59:14.252274+0000","last_active":"2024-01-20T08:59:14.252254+0000","last_peered":"2024-01-20T08:59:14.252254+0000","last_clean":"2024-01-20T08:59:14.252254+0000","last_became_active":"2024-01-20T08:31:10.278646+0000","last_became_peered":"2024-01-20T08:31:10.278646+0000","last_unstale":"2024-01-20T08:59:14.252254+0000","last_undegraded":"2024-01-20T08:59:14.252254+0000","last_fullsized":"2024-01-20T08:59:14.252254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-01-20T08:59:14.252192+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-01-20T08:59:14.252192+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.252192+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:47.209255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"27'44","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.500933+0000","last_change":"2024-01-20T08:59:14.500953+0000","last_active":"2024-01-20T08:59:14.500933+0000","last_peered":"2024-01-20T08:59:14.500933+0000","last_clean":"2024-01-20T08:59:14.500933+0000","last_became_active":"2024-01-20T08:31:10.260277+0000","last_became_peered":"2024-01-20T08:31:10.260277+0000","last_unstale":"2024-01-20T08:59:14.500933+0000","last_undegraded":"2024-01-20T08:59:14.500933+0000","last_fullsized":"2024-01-20T08:59:14.500933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'44","last_scrub_stamp":"2024-01-20T08:59:14.500872+0000","last_deep_scrub":"27'44","last_deep_scrub_stamp":"2024-01-20T08:59:14.500872+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.500872+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:10:40.652145+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'35","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.987389+0000","last_change":"2024-01-20T08:59:18.987428+0000","last_active":"2024-01-20T08:59:18.987389+0000","last_peered":"2024-01-20T08:59:18.987389+0000","last_clean":"2024-01-20T08:59:18.987389+0000","last_became_active":"2024-01-20T08:31:10.259962+0000","last_became_peered":"2024-01-20T08:31:10.259962+0000","last_unstale":"2024-01-20T08:59:18.987389+0000","last_undegraded":"2024-01-20T08:59:18.987389+0000","last_fullsized":"2024-01-20T08:59:18.987389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'35","last_scrub_stamp":"2024-01-20T08:59:18.987295+0000","last_deep_scrub":"27'35","last_deep_scrub_stamp":"2024-01-20T08:59:18.987295+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.987295+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:47:38.830913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"27'27","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.371846+0000","last_change":"2024-01-20T08:59:16.371846+0000","last_active":"2024-01-20T08:59:16.371846+0000","last_peered":"2024-01-20T08:59:16.371846+0000","last_clean":"2024-01-20T08:59:16.371846+0000","last_became_active":"2024-01-20T08:31:10.287299+0000","last_became_peered":"2024-01-20T08:31:10.287299+0000","last_unstale":"2024-01-20T08:59:16.371846+0000","last_undegraded":"2024-01-20T08:59:16.371846+0000","last_fullsized":"2024-01-20T08:59:16.371846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-01-20T08:59:16.371699+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-01-20T08:59:16.371699+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.371699+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:13:18.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'31","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.349087+0000","last_change":"2024-01-20T08:59:12.349087+0000","last_active":"2024-01-20T08:59:12.349087+0000","last_peered":"2024-01-20T08:59:12.349087+0000","last_clean":"2024-01-20T08:59:12.349087+0000","last_became_active":"2024-01-20T08:31:10.284647+0000","last_became_peered":"2024-01-20T08:31:10.284647+0000","last_unstale":"2024-01-20T08:59:12.349087+0000","last_undegraded":"2024-01-20T08:59:12.349087+0000","last_fullsized":"2024-01-20T08:59:12.349087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:12.349009+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:12.349009+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.349009+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:42:47.241016+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"27'44","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.029595+0000","last_change":"2024-01-20T08:59:10.029595+0000","last_active":"2024-01-20T08:59:10.029595+0000","last_peered":"2024-01-20T08:59:10.029595+0000","last_clean":"2024-01-20T08:59:10.029595+0000","last_became_active":"2024-01-20T08:31:10.234035+0000","last_became_peered":"2024-01-20T08:31:10.234035+0000","last_unstale":"2024-01-20T08:59:10.029595+0000","last_undegraded":"2024-01-20T08:59:10.029595+0000","last_fullsized":"2024-01-20T08:59:10.029595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'44","last_scrub_stamp":"2024-01-20T08:59:10.029487+0000","last_deep_scrub":"27'44","last_deep_scrub_stamp":"2024-01-20T08:59:10.029487+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.029487+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:37:12.065884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"27'34","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:20.992797+0000","last_change":"2024-01-20T08:59:20.992797+0000","last_active":"2024-01-20T08:59:20.992797+0000","last_peered":"2024-01-20T08:59:20.992797+0000","last_clean":"2024-01-20T08:59:20.992797+0000","last_became_active":"2024-01-20T08:31:10.278697+0000","last_became_peered":"2024-01-20T08:31:10.278697+0000","last_unstale":"2024-01-20T08:59:20.992797+0000","last_undegraded":"2024-01-20T08:59:20.992797+0000","last_fullsized":"2024-01-20T08:59:20.992797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:20.992690+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:20.992690+0000","last_clean_scrub_stamp":"2024-01-20T08:59:20.992690+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:15:39.303881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"27'34","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:24.718885+0000","last_change":"2024-01-20T08:59:24.718929+0000","last_active":"2024-01-20T08:59:24.718885+0000","last_peered":"2024-01-20T08:59:24.718885+0000","last_clean":"2024-01-20T08:59:24.718885+0000","last_became_active":"2024-01-20T08:31:10.263553+0000","last_became_peered":"2024-01-20T08:31:10.263553+0000","last_unstale":"2024-01-20T08:59:24.718885+0000","last_undegraded":"2024-01-20T08:59:24.718885+0000","last_fullsized":"2024-01-20T08:59:24.718885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:24.718791+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:24.718791+0000","last_clean_scrub_stamp":"2024-01-20T08:59:24.718791+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:11:24.369143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"27'42","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.048893+0000","last_change":"2024-01-20T08:59:17.048937+0000","last_active":"2024-01-20T08:59:17.048893+0000","last_peered":"2024-01-20T08:59:17.048893+0000","last_clean":"2024-01-20T08:59:17.048893+0000","last_became_active":"2024-01-20T08:31:10.259571+0000","last_became_peered":"2024-01-20T08:31:10.259571+0000","last_unstale":"2024-01-20T08:59:17.048893+0000","last_undegraded":"2024-01-20T08:59:17.048893+0000","last_fullsized":"2024-01-20T08:59:17.048893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:17.048777+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:17.048777+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.048777+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:02:46.925145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"27'38","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.894238+0000","last_change":"2024-01-20T08:59:11.894275+0000","last_active":"2024-01-20T08:59:11.894238+0000","last_peered":"2024-01-20T08:59:11.894238+0000","last_clean":"2024-01-20T08:59:11.894238+0000","last_became_active":"2024-01-20T08:31:10.284469+0000","last_became_peered":"2024-01-20T08:31:10.284469+0000","last_unstale":"2024-01-20T08:59:11.894238+0000","last_undegraded":"2024-01-20T08:59:11.894238+0000","last_fullsized":"2024-01-20T08:59:11.894238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'38","last_scrub_stamp":"2024-01-20T08:59:11.894159+0000","last_deep_scrub":"27'38","last_deep_scrub_stamp":"2024-01-20T08:59:11.894159+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.894159+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:23:22.695818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'47","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.725765+0000","last_change":"2024-01-20T08:59:13.725788+0000","last_active":"2024-01-20T08:59:13.725765+0000","last_peered":"2024-01-20T08:59:13.725765+0000","last_clean":"2024-01-20T08:59:13.725765+0000","last_became_active":"2024-01-20T08:31:10.191132+0000","last_became_peered":"2024-01-20T08:31:10.191132+0000","last_unstale":"2024-01-20T08:59:13.725765+0000","last_undegraded":"2024-01-20T08:59:13.725765+0000","last_fullsized":"2024-01-20T08:59:13.725765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-01-20T08:59:13.725705+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-01-20T08:59:13.725705+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.725705+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:10:43.766906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"27'41","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:17.702383+0000","last_change":"2024-01-20T08:59:17.702422+0000","last_active":"2024-01-20T08:59:17.702383+0000","last_peered":"2024-01-20T08:59:17.702383+0000","last_clean":"2024-01-20T08:59:17.702383+0000","last_became_active":"2024-01-20T08:31:10.277025+0000","last_became_peered":"2024-01-20T08:31:10.277025+0000","last_unstale":"2024-01-20T08:59:17.702383+0000","last_undegraded":"2024-01-20T08:59:17.702383+0000","last_fullsized":"2024-01-20T08:59:17.702383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:17.702301+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:17.702301+0000","last_clean_scrub_stamp":"2024-01-20T08:59:17.702301+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:37:33.977115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"27'34","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.463984+0000","last_change":"2024-01-20T08:59:10.464006+0000","last_active":"2024-01-20T08:59:10.463984+0000","last_peered":"2024-01-20T08:59:10.463984+0000","last_clean":"2024-01-20T08:59:10.463984+0000","last_became_active":"2024-01-20T08:31:10.236416+0000","last_became_peered":"2024-01-20T08:31:10.236416+0000","last_unstale":"2024-01-20T08:59:10.463984+0000","last_undegraded":"2024-01-20T08:59:10.463984+0000","last_fullsized":"2024-01-20T08:59:10.463984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-01-20T08:59:10.463920+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-01-20T08:59:10.463920+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.463920+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:31:57.709584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"27'45","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:13.255978+0000","last_change":"2024-01-20T08:59:13.256000+0000","last_active":"2024-01-20T08:59:13.255978+0000","last_peered":"2024-01-20T08:59:13.255978+0000","last_clean":"2024-01-20T08:59:13.255978+0000","last_became_active":"2024-01-20T08:31:10.236196+0000","last_became_peered":"2024-01-20T08:31:10.236196+0000","last_unstale":"2024-01-20T08:59:13.255978+0000","last_undegraded":"2024-01-20T08:59:13.255978+0000","last_fullsized":"2024-01-20T08:59:13.255978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-01-20T08:59:13.255913+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-01-20T08:59:13.255913+0000","last_clean_scrub_stamp":"2024-01-20T08:59:13.255913+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:27:17.393393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"27'38","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:06.446007+0000","last_change":"2024-01-20T08:59:06.446007+0000","last_active":"2024-01-20T08:59:06.446007+0000","last_peered":"2024-01-20T08:59:06.446007+0000","last_clean":"2024-01-20T08:59:06.446007+0000","last_became_active":"2024-01-20T08:31:10.235948+0000","last_became_peered":"2024-01-20T08:31:10.235948+0000","last_unstale":"2024-01-20T08:59:06.446007+0000","last_undegraded":"2024-01-20T08:59:06.446007+0000","last_fullsized":"2024-01-20T08:59:06.446007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'38","last_scrub_stamp":"2024-01-20T08:59:06.445911+0000","last_deep_scrub":"27'38","last_deep_scrub_stamp":"2024-01-20T08:59:06.445911+0000","last_clean_scrub_stamp":"2024-01-20T08:59:06.445911+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:32:02.791907+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"27'42","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.491798+0000","last_change":"2024-01-20T08:59:11.491823+0000","last_active":"2024-01-20T08:59:11.491798+0000","last_peered":"2024-01-20T08:59:11.491798+0000","last_clean":"2024-01-20T08:59:11.491798+0000","last_became_active":"2024-01-20T08:31:10.235355+0000","last_became_peered":"2024-01-20T08:31:10.235355+0000","last_unstale":"2024-01-20T08:59:11.491798+0000","last_undegraded":"2024-01-20T08:59:11.491798+0000","last_fullsized":"2024-01-20T08:59:11.491798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'42","last_scrub_stamp":"2024-01-20T08:59:11.491731+0000","last_deep_scrub":"27'42","last_deep_scrub_stamp":"2024-01-20T08:59:11.491731+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.491731+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T10:26:49.661101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"27'39","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.048545+0000","last_change":"2024-01-20T08:59:16.048566+0000","last_active":"2024-01-20T08:59:16.048545+0000","last_peered":"2024-01-20T08:59:16.048545+0000","last_clean":"2024-01-20T08:59:16.048545+0000","last_became_active":"2024-01-20T08:31:10.259499+0000","last_became_peered":"2024-01-20T08:31:10.259499+0000","last_unstale":"2024-01-20T08:59:16.048545+0000","last_undegraded":"2024-01-20T08:59:16.048545+0000","last_fullsized":"2024-01-20T08:59:16.048545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'39","last_scrub_stamp":"2024-01-20T08:59:16.048485+0000","last_deep_scrub":"27'39","last_deep_scrub_stamp":"2024-01-20T08:59:16.048485+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.048485+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T19:36:47.482687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"27'36","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.389145+0000","last_change":"2024-01-20T08:59:14.389145+0000","last_active":"2024-01-20T08:59:14.389145+0000","last_peered":"2024-01-20T08:59:14.389145+0000","last_clean":"2024-01-20T08:59:14.389145+0000","last_became_active":"2024-01-20T08:31:10.287027+0000","last_became_peered":"2024-01-20T08:31:10.287027+0000","last_unstale":"2024-01-20T08:59:14.389145+0000","last_undegraded":"2024-01-20T08:59:14.389145+0000","last_fullsized":"2024-01-20T08:59:14.389145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:14.389061+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:14.389061+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.389061+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:56.661649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"27'37","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.354770+0000","last_change":"2024-01-20T08:59:18.354808+0000","last_active":"2024-01-20T08:59:18.354770+0000","last_peered":"2024-01-20T08:59:18.354770+0000","last_clean":"2024-01-20T08:59:18.354770+0000","last_became_active":"2024-01-20T08:31:10.287642+0000","last_became_peered":"2024-01-20T08:31:10.287642+0000","last_unstale":"2024-01-20T08:59:18.354770+0000","last_undegraded":"2024-01-20T08:59:18.354770+0000","last_fullsized":"2024-01-20T08:59:18.354770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:18.354671+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:18.354671+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.354671+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:04:39.530834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'28","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:18.742962+0000","last_change":"2024-01-20T08:59:18.742983+0000","last_active":"2024-01-20T08:59:18.742962+0000","last_peered":"2024-01-20T08:59:18.742962+0000","last_clean":"2024-01-20T08:59:18.742962+0000","last_became_active":"2024-01-20T08:31:10.278588+0000","last_became_peered":"2024-01-20T08:31:10.278588+0000","last_unstale":"2024-01-20T08:59:18.742962+0000","last_undegraded":"2024-01-20T08:59:18.742962+0000","last_fullsized":"2024-01-20T08:59:18.742962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-01-20T08:59:18.742901+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-01-20T08:59:18.742901+0000","last_clean_scrub_stamp":"2024-01-20T08:59:18.742901+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:20:05.652480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"27'31","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:14.800935+0000","last_change":"2024-01-20T08:59:14.800956+0000","last_active":"2024-01-20T08:59:14.800935+0000","last_peered":"2024-01-20T08:59:14.800935+0000","last_clean":"2024-01-20T08:59:14.800935+0000","last_became_active":"2024-01-20T08:31:10.234379+0000","last_became_peered":"2024-01-20T08:31:10.234379+0000","last_unstale":"2024-01-20T08:59:14.800935+0000","last_undegraded":"2024-01-20T08:59:14.800935+0000","last_fullsized":"2024-01-20T08:59:14.800935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'31","last_scrub_stamp":"2024-01-20T08:59:14.800871+0000","last_deep_scrub":"27'31","last_deep_scrub_stamp":"2024-01-20T08:59:14.800871+0000","last_clean_scrub_stamp":"2024-01-20T08:59:14.800871+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T17:59:50.527407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"27'48","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:12.790132+0000","last_change":"2024-01-20T08:59:12.790168+0000","last_active":"2024-01-20T08:59:12.790132+0000","last_peered":"2024-01-20T08:59:12.790132+0000","last_clean":"2024-01-20T08:59:12.790132+0000","last_became_active":"2024-01-20T08:31:10.195655+0000","last_became_peered":"2024-01-20T08:31:10.195655+0000","last_unstale":"2024-01-20T08:59:12.790132+0000","last_undegraded":"2024-01-20T08:59:12.790132+0000","last_fullsized":"2024-01-20T08:59:12.790132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-01-20T08:59:12.790039+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-01-20T08:59:12.790039+0000","last_clean_scrub_stamp":"2024-01-20T08:59:12.790039+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:04:33.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"27'37","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.870735+0000","last_change":"2024-01-20T08:59:10.870757+0000","last_active":"2024-01-20T08:59:10.870735+0000","last_peered":"2024-01-20T08:59:10.870735+0000","last_clean":"2024-01-20T08:59:10.870735+0000","last_became_active":"2024-01-20T08:31:10.281025+0000","last_became_peered":"2024-01-20T08:31:10.281025+0000","last_unstale":"2024-01-20T08:59:10.870735+0000","last_undegraded":"2024-01-20T08:59:10.870735+0000","last_fullsized":"2024-01-20T08:59:10.870735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:10.870674+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:10.870674+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.870674+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:32:28.888081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"27'29","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:10.056411+0000","last_change":"2024-01-20T08:59:10.056470+0000","last_active":"2024-01-20T08:59:10.056411+0000","last_peered":"2024-01-20T08:59:10.056411+0000","last_clean":"2024-01-20T08:59:10.056411+0000","last_became_active":"2024-01-20T08:31:10.263421+0000","last_became_peered":"2024-01-20T08:31:10.263421+0000","last_unstale":"2024-01-20T08:59:10.056411+0000","last_undegraded":"2024-01-20T08:59:10.056411+0000","last_fullsized":"2024-01-20T08:59:10.056411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-01-20T08:59:10.056323+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-01-20T08:59:10.056323+0000","last_clean_scrub_stamp":"2024-01-20T08:59:10.056323+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T15:03:24.090240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'36","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:05.751587+0000","last_change":"2024-01-20T08:59:05.751608+0000","last_active":"2024-01-20T08:59:05.751587+0000","last_peered":"2024-01-20T08:59:05.751587+0000","last_clean":"2024-01-20T08:59:05.751587+0000","last_became_active":"2024-01-20T08:31:10.197456+0000","last_became_peered":"2024-01-20T08:31:10.197456+0000","last_unstale":"2024-01-20T08:59:05.751587+0000","last_undegraded":"2024-01-20T08:59:05.751587+0000","last_fullsized":"2024-01-20T08:59:05.751587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-01-20T08:59:05.751528+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-01-20T08:59:05.751528+0000","last_clean_scrub_stamp":"2024-01-20T08:59:05.751528+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:40:23.102642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"27'37","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:08.081918+0000","last_change":"2024-01-20T08:59:08.081958+0000","last_active":"2024-01-20T08:59:08.081918+0000","last_peered":"2024-01-20T08:59:08.081918+0000","last_clean":"2024-01-20T08:59:08.081918+0000","last_became_active":"2024-01-20T08:31:10.192284+0000","last_became_peered":"2024-01-20T08:31:10.192284+0000","last_unstale":"2024-01-20T08:59:08.081918+0000","last_undegraded":"2024-01-20T08:59:08.081918+0000","last_fullsized":"2024-01-20T08:59:08.081918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-01-20T08:59:08.081835+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-01-20T08:59:08.081835+0000","last_clean_scrub_stamp":"2024-01-20T08:59:08.081835+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T20:23:18.883494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"27'30","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:16.726790+0000","last_change":"2024-01-20T08:59:16.726824+0000","last_active":"2024-01-20T08:59:16.726790+0000","last_peered":"2024-01-20T08:59:16.726790+0000","last_clean":"2024-01-20T08:59:16.726790+0000","last_became_active":"2024-01-20T08:31:10.262966+0000","last_became_peered":"2024-01-20T08:31:10.262966+0000","last_unstale":"2024-01-20T08:59:16.726790+0000","last_undegraded":"2024-01-20T08:59:16.726790+0000","last_fullsized":"2024-01-20T08:59:16.726790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-01-20T08:59:16.726728+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-01-20T08:59:16.726728+0000","last_clean_scrub_stamp":"2024-01-20T08:59:16.726728+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T11:52:46.998792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:07.101535+0000","last_change":"2024-01-20T08:59:07.101535+0000","last_active":"2024-01-20T08:59:07.101535+0000","last_peered":"2024-01-20T08:59:07.101535+0000","last_clean":"2024-01-20T08:59:07.101535+0000","last_became_active":"2024-01-20T08:31:10.190884+0000","last_became_peered":"2024-01-20T08:31:10.190884+0000","last_unstale":"2024-01-20T08:59:07.101535+0000","last_undegraded":"2024-01-20T08:59:07.101535+0000","last_fullsized":"2024-01-20T08:59:07.101535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-01-20T08:59:07.101443+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-01-20T08:59:07.101443+0000","last_clean_scrub_stamp":"2024-01-20T08:59:07.101443+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T09:32:25.118851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'41","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:11.008640+0000","last_change":"2024-01-20T08:59:11.008662+0000","last_active":"2024-01-20T08:59:11.008640+0000","last_peered":"2024-01-20T08:59:11.008640+0000","last_clean":"2024-01-20T08:59:11.008640+0000","last_became_active":"2024-01-20T08:31:10.262251+0000","last_became_peered":"2024-01-20T08:31:10.262251+0000","last_unstale":"2024-01-20T08:59:11.008640+0000","last_undegraded":"2024-01-20T08:59:11.008640+0000","last_fullsized":"2024-01-20T08:59:11.008640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'41","last_scrub_stamp":"2024-01-20T08:59:11.008572+0000","last_deep_scrub":"27'41","last_deep_scrub_stamp":"2024-01-20T08:59:11.008572+0000","last_clean_scrub_stamp":"2024-01-20T08:59:11.008572+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T18:50:53.281160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"27'33","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:22.013088+0000","last_change":"2024-01-20T08:59:22.013111+0000","last_active":"2024-01-20T08:59:22.013088+0000","last_peered":"2024-01-20T08:59:22.013088+0000","last_clean":"2024-01-20T08:59:22.013088+0000","last_became_active":"2024-01-20T08:31:10.284581+0000","last_became_peered":"2024-01-20T08:31:10.284581+0000","last_unstale":"2024-01-20T08:59:22.013088+0000","last_undegraded":"2024-01-20T08:59:22.013088+0000","last_fullsized":"2024-01-20T08:59:22.013088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'33","last_scrub_stamp":"2024-01-20T08:59:22.013017+0000","last_deep_scrub":"27'33","last_deep_scrub_stamp":"2024-01-20T08:59:22.013017+0000","last_clean_scrub_stamp":"2024-01-20T08:59:22.013017+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T14:18:30.736742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"27'26","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-01-20T08:59:04.749273+0000","last_change":"2024-01-20T08:59:04.749310+0000","last_active":"2024-01-20T08:59:04.749273+0000","last_peered":"2024-01-20T08:59:04.749273+0000","last_clean":"2024-01-20T08:59:04.749273+0000","last_became_active":"2024-01-20T08:31:10.233770+0000","last_became_peered":"2024-01-20T08:31:10.233770+0000","last_unstale":"2024-01-20T08:59:04.749273+0000","last_undegraded":"2024-01-20T08:59:04.749273+0000","last_fullsized":"2024-01-20T08:59:04.749273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-01-20T08:59:04.749186+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-01-20T08:59:04.749186+0000","last_clean_scrub_stamp":"2024-01-20T08:59:04.749186+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-21T13:51:15.202067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":308552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4420,"ondisk_log_size":4420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39037201,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":967,"num_read_kb":1028,"num_write":8593,"num_write_kb":43222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78323712,"data_stored":78074402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8212,"ondisk_log_size":8212,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509794,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39732,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332108,"statfs":{"total":96636764160,"available":96596078592,"internally_reserved":0,"allocated":4501504,"data_stored":4315319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.39600000000000002,"5min":0.29199999999999998,"15min":0.23899999999999999},"max":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.67100000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.81999999999999995},"last":0.41499999999999998}]},{"osd":1,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.35299999999999998,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.67900000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.28999999999999998},"max":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.67500000000000004},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.42099999999999999,"5min":0.30099999999999999,"15min":0.28399999999999997},"max":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.70899999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.46500000000000002,"15min":0.46999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.629,"5min":0.68500000000000005,"15min":0.72699999999999998},"last":0.438}]},{"osd":3,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.34999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.60899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30299999999999999},"max":{"1min":0.56799999999999995,"5min":0.629,"15min":0.74199999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.312,"5min":0.248,"15min":0.23799999999999999},"max":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.76700000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.30399999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67200000000000004,"5min":0.75800000000000001,"15min":0.876},"last":0.29999999999999999}]},{"osd":5,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.439,"15min":0.45200000000000001},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58199999999999996,"5min":0.67400000000000004,"15min":0.81000000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68700000000000006,"5min":0.73299999999999998,"15min":0.79400000000000004},"last":0.40400000000000003}]},{"osd":7,"last update":"Sat Jan 20 08:59:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.62,"5min":0.78400000000000003,"15min":0.90900000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.50600000000000001,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.85299999999999998},"last":0.32200000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509794,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67036,"kb_used_data":13652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94304804,"statfs":{"total":96636764160,"available":96568119296,"internally_reserved":0,"allocated":13979648,"data_stored":13773597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.29499999999999998,"15min":0.29599999999999999},"min":{"1min":0.22500000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.72599999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19400000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.58799999999999997},"last":0.40999999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.28499999999999998,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.61799999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.377}]},{"osd":3,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.27500000000000002,"5min":0.219,"15min":0.21099999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.79600000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.371},"min":{"1min":0.32800000000000001,"5min":0.245,"15min":0.24299999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.69499999999999995},"last":0.39800000000000002}]},{"osd":4,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.76200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.38500000000000001,"15min":0.30099999999999999},"max":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.80100000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.317,"15min":0.317},"max":{"1min":0.79100000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.91400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.878,"15min":0.878},"last":0.497}]},{"osd":7,"last update":"Sat Jan 20 08:59:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.438,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54500000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542496,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70924,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300916,"statfs":{"total":96636764160,"available":96564137984,"internally_reserved":0,"allocated":13701120,"data_stored":13502280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45700000000000002,"15min":0.46200000000000002},"min":{"1min":0.34100000000000003,"5min":0.24299999999999999,"15min":0.23799999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46300000000000002,"15min":0.45600000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"last":0.64800000000000002}]},{"osd":1,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67900000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46500000000000002,"15min":0.46000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.28199999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.66000000000000003},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.25600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.72899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.37,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45100000000000001}]},{"osd":3,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.311},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.76700000000000002},"last":0.629},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.80900000000000005},"last":0.62}]},{"osd":4,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.26600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.625,"5min":0.754,"15min":0.80300000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.46600000000000003,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.27100000000000002,"15min":0.26500000000000001},"max":{"1min":0.62,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.498}]},{"osd":6,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.44400000000000001,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56100000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.379},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.81599999999999995},"last":0.60399999999999998}]},{"osd":7,"last update":"Sat Jan 20 08:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47799999999999998,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69399999999999995,"5min":0.70099999999999996,"15min":0.80300000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51400000000000001,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.85799999999999998},"last":0.47099999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542495,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66068,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305772,"statfs":{"total":96636764160,"available":96569110528,"internally_reserved":0,"allocated":8728576,"data_stored":8508591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.34599999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.24099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47799999999999998}]},{"osd":1,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.35199999999999998,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45700000000000002},"min":{"1min":0.31,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69799999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.37,"5min":0.31,"15min":0.26500000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.502},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.377,"5min":0.31,"15min":0.28599999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66700000000000004},"last":0.57899999999999996}]},{"osd":3,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.33300000000000002,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.40899999999999997,"15min":0.42699999999999999},"min":{"1min":0.254,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58999999999999997,"5min":0.71899999999999997,"15min":0.73999999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.47799999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.254},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.49399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.495},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Jan 20 08:59:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67700000000000005,"5min":0.73999999999999999,"15min":0.79100000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.79000000000000004},"last":0.52800000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542496,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43824,"kb_used_data":8488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328016,"statfs":{"total":96636764160,"available":96591888384,"internally_reserved":0,"allocated":8691712,"data_stored":8506821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:59:38 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.318,"15min":0.307},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.309},"min":{"1min":0.24399999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":0.45900000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Jan 20 08:59:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.34200000000000003},"min":{"1min":0.26900000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.439},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.35099999999999998,"15min":0.34599999999999997},"min":{"1min":0.27000000000000002,"5min":0.23400000000000001,"15min":0.22},"max":{"1min":0.41999999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.33200000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:59:29 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33400000000000002},"min":{"1min":0.23100000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.48499999999999999,"5min":0.55900000000000005,"15min":0.60799999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.251,"5min":0.251,"15min":0.247},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47499999999999998}]},{"osd":4,"last update":"Sat Jan 20 08:59:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.82099999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.747,"5min":0.747,"15min":0.79800000000000004},"last":0.61799999999999999}]},{"osd":5,"last update":"Sat Jan 20 08:59:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52000000000000002,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.316},"max":{"1min":0.65600000000000003,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.374,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66400000000000003,"5min":0.83799999999999997,"15min":0.86699999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Sat Jan 20 08:59:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.77600000000000002,"15min":0.80500000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69599999999999995,"5min":0.80300000000000005,"15min":0.83499999999999996},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Jan 20 08:59:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.33800000000000002},"max":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.79300000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542497,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9289728,"data_stored":9097863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30099999999999999,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.55700000000000005,"15min":0.59499999999999997},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.32000000000000001,"15min":0.31},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20000000000000001},"max":{"1min":0.40600000000000003,"5min":0.45500000000000002,"15min":0.54400000000000004},"last":0.22600000000000001}]},{"osd":2,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.308,"15min":0.314},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.35399999999999998,"5min":0.47999999999999998,"15min":0.49199999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":0.45800000000000002,"5min":0.624,"15min":0.624},"last":0.27100000000000002}]},{"osd":3,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.59399999999999997,"15min":0.63},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.44700000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.31}]},{"osd":4,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.27900000000000003},"max":{"1min":0.80700000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54900000000000004}]},{"osd":5,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.876},"last":0.629}]},{"osd":6,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":0.873,"15min":0.873},"last":0.61199999999999999}]},{"osd":7,"last update":"Sat Jan 20 08:58:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56899999999999995}]}]},{"osd":7,"up_from":13,"seq":55834575200,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63212,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308628,"statfs":{"total":96636764160,"available":96572035072,"internally_reserved":0,"allocated":5804032,"data_stored":5592459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.68999999999999995},"last":0.53900000000000003}]},{"osd":3,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.54200000000000004,"5min":0.65100000000000002,"15min":0.73499999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.37,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.78000000000000003},"last":0.498}]},{"osd":4,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23000000000000001},"max":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.75800000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26600000000000001},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.79900000000000004},"last":0.377}]},{"osd":5,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.76400000000000001,"15min":0.77800000000000002},"last":0.40799999999999997}]},{"osd":6,"last update":"Sat Jan 20 08:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.46300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575200,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61772,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310068,"statfs":{"total":96636764160,"available":96573509632,"internally_reserved":0,"allocated":17108992,"data_stored":16897992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.314,"15min":0.32100000000000001},"min":{"1min":0.26000000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.64300000000000002},"last":0.314},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.312,"15min":0.308},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.51200000000000001},"last":0.27400000000000002}]},{"osd":1,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32200000000000001,"15min":0.32100000000000001},"min":{"1min":0.24199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.65000000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.34300000000000003},"min":{"1min":0.26900000000000002,"5min":0.22,"15min":0.214},"max":{"1min":0.56899999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35299999999999998}]},{"osd":2,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.34899999999999998,"15min":0.35699999999999998},"min":{"1min":0.28199999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46400000000000002,"5min":0.52500000000000002,"15min":0.625},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.378,"15min":0.377},"min":{"1min":0.251,"5min":0.22,"15min":0.22},"max":{"1min":0.55900000000000005,"5min":0.66300000000000003,"15min":0.71599999999999997},"last":0.376}]},{"osd":4,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.72399999999999998,"5min":0.78500000000000003,"15min":0.94399999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51600000000000001,"15min":0.52200000000000002},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.85499999999999998},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.95699999999999996},"last":0.503},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.82199999999999995},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.55300000000000005},"min":{"1min":0.39500000000000002,"5min":0.36799999999999999,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.80900000000000005,"15min":0.85699999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.34499999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.96699999999999997},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Jan 20 08:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77800000000000002,"5min":0.79600000000000004,"15min":0.84099999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.90700000000000003},"last":0.56999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13692928,"data_stored":13656124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8409088,"data_stored":8390022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":8389348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16822272,"data_stored":16780519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8441856,"data_stored":8391118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12820480,"data_stored":12794439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4214784,"data_stored":4197846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":5474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-20T08:59:51.352 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-20T08:59:51.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.483+0000 7f8b45215640 1 Processor -- start 2024-01-20T08:59:51.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.483+0000 7f8b45215640 1 -- start start 2024-01-20T08:59:51.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b45215640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b0c60 0x7f8b4006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:51.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b45215640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b400709e0 con 0x7f8b400b1220 2024-01-20T08:59:51.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b45215640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4015a130 con 0x7f8b40153880 2024-01-20T08:59:51.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b45215640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401595f0 con 0x7f8b400b0c60 2024-01-20T08:59:51.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b3ed9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8b40153880 0x7f8b4015cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:40342/0 (socket says 172.21.15.87:40342) 2024-01-20T08:59:51.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b3ed9d640 1 -- 172.21.15.87:0/3178824184 learned_addr learned my addr 172.21.15.87:0/3178824184 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:51.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.484+0000 7f8b3f59e640 1 --2- 172.21.15.87:0/3178824184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b0c60 0x7f8b4006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:51.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.487+0000 7f8b3dd9b640 1 -- 172.21.15.87:0/3178824184 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735501176 0 0) 0x7f8b4015a130 con 0x7f8b40153880 2024-01-20T08:59:51.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.487+0000 7f8b3dd9b640 1 -- 172.21.15.87:0/3178824184 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b28003520 con 0x7f8b40153880 2024-01-20T08:59:51.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.487+0000 7f8b3f59e640 1 -- 172.21.15.87:0/3178824184 >> v1:172.21.15.116:6789/0 conn(0x7f8b40153880 legacy=0x7f8b4015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.487+0000 7f8b3f59e640 1 -- 172.21.15.87:0/3178824184 >> v1:172.21.15.87:6789/0 conn(0x7f8b400b1220 legacy=0x7f8b4006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.487+0000 7f8b3f59e640 1 -- 172.21.15.87:0/3178824184 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40070720 con 0x7f8b400b0c60 2024-01-20T08:59:51.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.491+0000 7f8b3f59e640 1 --2- 172.21.15.87:0/3178824184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b0c60 0x7f8b4006df10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009dc0 tx=0x7f8b3802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca55bab35b55326c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:51.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.497+0000 7f8b3dd9b640 1 -- 172.21.15.87:0/3178824184 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3802cd50 con 0x7f8b400b0c60 2024-01-20T08:59:51.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.497+0000 7f8b3dd9b640 1 -- 172.21.15.87:0/3178824184 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38034440 con 0x7f8b400b0c60 2024-01-20T08:59:51.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.497+0000 7f8b3dd9b640 1 -- 172.21.15.87:0/3178824184 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380347d0 con 0x7f8b400b0c60 2024-01-20T08:59:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 -- 172.21.15.87:0/3178824184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b0c60 msgr2=0x7f8b4006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 --2- 172.21.15.87:0/3178824184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b0c60 0x7f8b4006df10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009dc0 tx=0x7f8b3802bfa0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 -- 172.21.15.87:0/3178824184 shutdown_connections 2024-01-20T08:59:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 --2- 172.21.15.87:0/3178824184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b0c60 0x7f8b4006df10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 -- 172.21.15.87:0/3178824184 >> 172.21.15.87:0/3178824184 conn(0x7f8b40072390 msgr2=0x7f8b4006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 -- 172.21.15.87:0/3178824184 shutdown_connections 2024-01-20T08:59:51.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.498+0000 7f8b45215640 1 -- 172.21.15.87:0/3178824184 wait complete. 2024-01-20T08:59:51.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.500+0000 7f8b45215640 1 Processor -- start 2024-01-20T08:59:51.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.500+0000 7f8b45215640 1 -- start start 2024-01-20T08:59:51.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.500+0000 7f8b45215640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b400b0c60 0x7f8b401f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.500+0000 7f8b45215640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 0x7f8b400c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.500+0000 7f8b45215640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b40153880 0x7f8b40150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.501+0000 7f8b45215640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400c38e0 con 0x7f8b40153880 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.501+0000 7f8b45215640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401efcb0 con 0x7f8b400b0c60 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.501+0000 7f8b45215640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401ef100 con 0x7f8b400b1220 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.501+0000 7f8b3e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 0x7f8b400c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.501+0000 7f8b3e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 0x7f8b400c18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:42206/0 (socket says 172.21.15.87:42206) 2024-01-20T08:59:51.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.501+0000 7f8b3e59c640 1 -- 172.21.15.87:0/898944718 learned_addr learned my addr 172.21.15.87:0/898944718 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:51.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.503+0000 7f8b3ed9d640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b400b0c60 0x7f8b401f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:51.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.505+0000 7f8b3e59c640 1 -- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b400b0c60 msgr2=0x7f8b401f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.505+0000 7f8b3e59c640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b400b0c60 0x7f8b401f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.505+0000 7f8b3e59c640 1 -- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b40153880 msgr2=0x7f8b40150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:51.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.505+0000 7f8b3e59c640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b40153880 0x7f8b40150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:51.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.505+0000 7f8b3e59c640 1 -- 172.21.15.87:0/898944718 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009a70 con 0x7f8b400b1220 2024-01-20T08:59:51.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.508+0000 7f8b3e59c640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 0x7f8b400c18f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400c430 tx=0x7f8b3400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.512+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340052d0 con 0x7f8b400b1220 2024-01-20T08:59:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.512+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34005450 con 0x7f8b400b1220 2024-01-20T08:59:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.512+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34007260 con 0x7f8b400b1220 2024-01-20T08:59:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.512+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b401f3570 con 0x7f8b400b1220 2024-01-20T08:59:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.512+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b401ef380 con 0x7f8b400b1220 2024-01-20T08:59:51.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.513+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b401efe60 con 0x7f8b400b1220 2024-01-20T08:59:51.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.533+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8b340073c0 con 0x7f8b400b1220 2024-01-20T08:59:51.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.533+0000 7f8b277fe640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8b2c06af50 0x7f8b2c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:51.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.534+0000 7f8b3ed9d640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8b2c06af50 0x7f8b2c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:51.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.534+0000 7f8b3ed9d640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8b2c06af50 0x7f8b2c06d410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b300011e0 tx=0x7f8b30009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:51.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.538+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f8b340581d0 con 0x7f8b400b1220 2024-01-20T08:59:51.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.774+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8b34053e10 con 0x7f8b400b1220 2024-01-20T08:59:51.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:51.950+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8b400709e0 con 0x7f8b400b1220 2024-01-20T08:59:52.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.134+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8b34053e10 con 0x7f8b400b1220 2024-01-20T08:59:52.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.134+0000 7f8b277fe640 1 -- 172.21.15.87:0/898944718 <== mon.2 v2:172.21.15.116:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8b400709e0 con 0x7f8b400b1220 2024-01-20T08:59:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8b2c06af50 msgr2=0x7f8b2c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8b2c06af50 0x7f8b2c06d410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b300011e0 tx=0x7f8b30009040 comp rx=0 tx=0).stop 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 msgr2=0x7f8b400c18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 0x7f8b400c18f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400c430 tx=0x7f8b3400c8f0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 shutdown_connections 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f8b2c06af50 0x7f8b2c06d410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b40153880 0x7f8b40150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b400b1220 0x7f8b400c18f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 --2- 172.21.15.87:0/898944718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b400b0c60 0x7f8b401f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 >> 172.21.15.87:0/898944718 conn(0x7f8b40072390 msgr2=0x7f8b4014ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 shutdown_connections 2024-01-20T08:59:52.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.136+0000 7f8b45215640 1 -- 172.21.15.87:0/898944718 wait complete. 2024-01-20T08:59:52.148 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-20T08:59:52.149 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:59:52.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.247+0000 7fe75fb83640 1 Processor -- start 2024-01-20T08:59:52.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.248+0000 7fe75fb83640 1 -- start start 2024-01-20T08:59:52.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.248+0000 7fe75fb83640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe758153aa0 0x7fe7580c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.248+0000 7fe75fb83640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7580ca2d0 0x7fe75815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.248+0000 7fe75fb83640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75815cc70 con 0x7fe75815d920 2024-01-20T08:59:52.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.248+0000 7fe75fb83640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe758153e80 con 0x7fe758153aa0 2024-01-20T08:59:52.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.248+0000 7fe75fb83640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75806b880 con 0x7fe7580ca2d0 2024-01-20T08:59:52.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.249+0000 7fe75e120640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe75815d920 0x7fe7580c88e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51400/0 (socket says 172.21.15.87:51400) 2024-01-20T08:59:52.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.249+0000 7fe75e120640 1 -- 172.21.15.87:0/1863279416 learned_addr learned my addr 172.21.15.87:0/1863279416 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:52.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.249+0000 7fe75d91f640 1 --2- 172.21.15.87:0/1863279416 >> v2:172.21.15.116:3300/0 conn(0x7fe758153aa0 0x7fe7580c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.249+0000 7fe75d11e640 1 --2- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7580ca2d0 0x7fe75815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.251+0000 7fe75c91d640 1 -- 172.21.15.87:0/1863279416 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40557688 0 0) 0x7fe75815cc70 con 0x7fe75815d920 2024-01-20T08:59:52.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.251+0000 7fe75c91d640 1 -- 172.21.15.87:0/1863279416 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe744003520 con 0x7fe75815d920 2024-01-20T08:59:52.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.252+0000 7fe75d91f640 1 -- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7580ca2d0 msgr2=0x7fe75815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:52.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.252+0000 7fe75d91f640 1 --2- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7580ca2d0 0x7fe75815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.252+0000 7fe75d91f640 1 -- 172.21.15.87:0/1863279416 >> v1:172.21.15.87:6789/0 conn(0x7fe75815d920 legacy=0x7fe7580c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.252+0000 7fe75d91f640 1 -- 172.21.15.87:0/1863279416 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758159c40 con 0x7fe758153aa0 2024-01-20T08:59:52.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.254+0000 7fe75d91f640 1 --2- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe758153aa0 0x7fe7580c7610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe754009d90 tx=0x7fe75402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62f73e3f207f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:52.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.257+0000 7fe75c91d640 1 -- 172.21.15.87:0/1863279416 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75402cce0 con 0x7fe758153aa0 2024-01-20T08:59:52.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.257+0000 7fe75c91d640 1 -- 172.21.15.87:0/1863279416 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe75402ce40 con 0x7fe758153aa0 2024-01-20T08:59:52.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.257+0000 7fe75c91d640 1 -- 172.21.15.87:0/1863279416 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754034680 con 0x7fe758153aa0 2024-01-20T08:59:52.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 -- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe758153aa0 msgr2=0x7fe7580c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe758153aa0 0x7fe7580c7610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe754009d90 tx=0x7fe75402bf30 comp rx=0 tx=0).stop 2024-01-20T08:59:52.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 -- 172.21.15.87:0/1863279416 shutdown_connections 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7580ca2d0 0x7fe75815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1863279416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe758153aa0 0x7fe7580c7610 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 -- 172.21.15.87:0/1863279416 >> 172.21.15.87:0/1863279416 conn(0x7fe758066ac0 msgr2=0x7fe75806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 -- 172.21.15.87:0/1863279416 shutdown_connections 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.258+0000 7fe75fb83640 1 -- 172.21.15.87:0/1863279416 wait complete. 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.259+0000 7fe75fb83640 1 Processor -- start 2024-01-20T08:59:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 -- start start 2024-01-20T08:59:52.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe758153aa0 0x7fe75814bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815d920 0x7fe7581ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 0x7fe7581ef820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75815bb10 con 0x7fe75815c2e0 2024-01-20T08:59:52.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580c9ec0 con 0x7fe75815d920 2024-01-20T08:59:52.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.260+0000 7fe75fb83640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75815bd20 con 0x7fe758153aa0 2024-01-20T08:59:52.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.261+0000 7fe75e120640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 0x7fe7581ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.261+0000 7fe75d11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815d920 0x7fe7581ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.261+0000 7fe75d91f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe758153aa0 0x7fe75814bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.261+0000 7fe75e120640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 0x7fe7581ef820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38386/0 (socket says 172.21.15.87:38386) 2024-01-20T08:59:52.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.261+0000 7fe75e120640 1 -- 172.21.15.87:0/1906061052 learned_addr learned my addr 172.21.15.87:0/1906061052 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:52.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.264+0000 7fe75e120640 1 -- 172.21.15.87:0/1906061052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe758153aa0 msgr2=0x7fe75814bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.264+0000 7fe75e120640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe758153aa0 0x7fe75814bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.264+0000 7fe75e120640 1 -- 172.21.15.87:0/1906061052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815d920 msgr2=0x7fe7581ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.264+0000 7fe75e120640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815d920 0x7fe7581ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.264+0000 7fe75e120640 1 -- 172.21.15.87:0/1906061052 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754009a70 con 0x7fe75815c2e0 2024-01-20T08:59:52.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.267+0000 7fe75e120640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 0x7fe7581ef820 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe750001eb0 tx=0x7fe75000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:52.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.269+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750005ce0 con 0x7fe75815c2e0 2024-01-20T08:59:52.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.269+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe758158ab0 con 0x7fe75815c2e0 2024-01-20T08:59:52.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.269+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe758159070 con 0x7fe75815c2e0 2024-01-20T08:59:52.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.270+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe758153300 con 0x7fe75815c2e0 2024-01-20T08:59:52.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.516+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe750007e80 con 0x7fe75815c2e0 2024-01-20T08:59:52.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.516+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750007270 con 0x7fe75815c2e0 2024-01-20T08:59:52.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.516+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe750007490 con 0x7fe75815c2e0 2024-01-20T08:59:52.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.517+0000 7fe74e7fc640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe73c06aed0 0x7fe73c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.517+0000 7fe75d91f640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe73c06aed0 0x7fe73c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.518+0000 7fe75d91f640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe73c06aed0 0x7fe73c06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe754036000 tx=0x7fe754030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:52.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.544+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fe750057d10 con 0x7fe75815c2e0 2024-01-20T08:59:52.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.544+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe750053fc0 con 0x7fe75815c2e0 2024-01-20T08:59:52.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.796+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe758074290 con 0x7fe75815c2e0 2024-01-20T08:59:52.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.808+0000 7fe74e7fc640 1 -- 172.21.15.87:0/1906061052 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6584 (secure 0 0 0) 0x7fe750017020 con 0x7fe75815c2e0 2024-01-20T08:59:52.810 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:52.810 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:45.461258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4290,"mds_2":4287,"mds_3":4284,"mds_4":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":4,"incarnation":17,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":12,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4287":{"gid":4287,"name":"c","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4290":{"gid":4290,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-20T08:59:52.810 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 18 2024-01-20T08:59:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe73c06aed0 msgr2=0x7fe73c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe73c06aed0 0x7fe73c06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe754036000 tx=0x7fe754030000 comp rx=0 tx=0).stop 2024-01-20T08:59:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 msgr2=0x7fe7581ef820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 0x7fe7581ef820 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe750001eb0 tx=0x7fe75000c820 comp rx=0 tx=0).stop 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 shutdown_connections 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7fe73c06aed0 0x7fe73c06d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75815c2e0 0x7fe7581ef820 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815d920 0x7fe7581ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 --2- 172.21.15.87:0/1906061052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe758153aa0 0x7fe75814bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 >> 172.21.15.87:0/1906061052 conn(0x7fe7580ca2d0 msgr2=0x7fe7580c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 shutdown_connections 2024-01-20T08:59:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.811+0000 7fe75fb83640 1 -- 172.21.15.87:0/1906061052 wait complete. 2024-01-20T08:59:52.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-20T08:59:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.925+0000 7f5e9919e640 1 Processor -- start 2024-01-20T08:59:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.926+0000 7f5e9919e640 1 -- start start 2024-01-20T08:59:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.926+0000 7f5e9919e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e94153910 0x7f5e940cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.926+0000 7f5e9919e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5e9415d5c0 0x7f5e9415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.926+0000 7f5e9919e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e94153cf0 con 0x7f5e9415d000 2024-01-20T08:59:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.926+0000 7f5e9919e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e940cc520 con 0x7f5e9415d5c0 2024-01-20T08:59:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.926+0000 7f5e9919e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e940c9a80 con 0x7f5e94153910 2024-01-20T08:59:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.927+0000 7f5e937fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5e9415d5c0 0x7f5e9415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.927+0000 7f5e93fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e94153910 0x7f5e940cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.927+0000 7f5e937fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5e9415d5c0 0x7f5e9415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38394/0 (socket says 172.21.15.87:38394) 2024-01-20T08:59:52.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.927+0000 7f5e937fe640 1 -- 172.21.15.87:0/3326292017 learned_addr learned my addr 172.21.15.87:0/3326292017 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:52.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.930+0000 7f5e937fe640 1 -- 172.21.15.87:0/3326292017 >> v2:172.21.15.116:3300/0 conn(0x7f5e94153910 msgr2=0x7f5e940cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:52.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.930+0000 7f5e937fe640 1 --2- 172.21.15.87:0/3326292017 >> v2:172.21.15.116:3300/0 conn(0x7f5e94153910 0x7f5e940cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.930+0000 7f5e937fe640 1 -- 172.21.15.87:0/3326292017 >> v1:172.21.15.116:6789/0 conn(0x7f5e9415d000 legacy=0x7f5e940c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.930+0000 7f5e937fe640 1 -- 172.21.15.87:0/3326292017 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940c9d60 con 0x7f5e9415d5c0 2024-01-20T08:59:52.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.932+0000 7f5e937fe640 1 --2- 172.21.15.87:0/3326292017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415cac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e9406d4f0 tx=0x7f5e7c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d68c6189da86e0c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:52.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.935+0000 7f5e92ffd640 1 -- 172.21.15.87:0/3326292017 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c02f850 con 0x7f5e9415d5c0 2024-01-20T08:59:52.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e92ffd640 1 -- 172.21.15.87:0/3326292017 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5e7c02f9b0 con 0x7f5e9415d5c0 2024-01-20T08:59:52.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e92ffd640 1 -- 172.21.15.87:0/3326292017 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c02fd60 con 0x7f5e9415d5c0 2024-01-20T08:59:52.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 -- 172.21.15.87:0/3326292017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e9415d5c0 msgr2=0x7f5e9415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 --2- 172.21.15.87:0/3326292017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415cac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e9406d4f0 tx=0x7f5e7c02ba30 comp rx=0 tx=0).stop 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 -- 172.21.15.87:0/3326292017 shutdown_connections 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 --2- 172.21.15.87:0/3326292017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415cac0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 --2- 172.21.15.87:0/3326292017 >> v2:172.21.15.116:3300/0 conn(0x7f5e94153910 0x7f5e940cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 -- 172.21.15.87:0/3326292017 >> 172.21.15.87:0/3326292017 conn(0x7f5e94071ce0 msgr2=0x7f5e940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 -- 172.21.15.87:0/3326292017 shutdown_connections 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.936+0000 7f5e9919e640 1 -- 172.21.15.87:0/3326292017 wait complete. 2024-01-20T08:59:52.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 Processor -- start 2024-01-20T08:59:52.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 -- start start 2024-01-20T08:59:52.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 0x7f5e941e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e9415d000 0x7f5e9414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9415b040 con 0x7f5e94153910 2024-01-20T08:59:52.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9415a750 con 0x7f5e9415d5c0 2024-01-20T08:59:52.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.938+0000 7f5e9919e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e941efdb0 con 0x7f5e9415d000 2024-01-20T08:59:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.939+0000 7f5e93fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 0x7f5e941e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.939+0000 7f5e9899d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.939+0000 7f5e937fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e9415d000 0x7f5e9414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.939+0000 7f5e93fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 0x7f5e941e52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38408/0 (socket says 172.21.15.87:38408) 2024-01-20T08:59:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.939+0000 7f5e93fff640 1 -- 172.21.15.87:0/4194542324 learned_addr learned my addr 172.21.15.87:0/4194542324 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:52.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.943+0000 7f5e93fff640 1 -- 172.21.15.87:0/4194542324 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e9415d000 msgr2=0x7f5e9414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.943+0000 7f5e93fff640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e9415d000 0x7f5e9414ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.943+0000 7f5e93fff640 1 -- 172.21.15.87:0/4194542324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e9415d5c0 msgr2=0x7f5e9415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:52.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.943+0000 7f5e93fff640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:52.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.943+0000 7f5e93fff640 1 -- 172.21.15.87:0/4194542324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0094d0 con 0x7f5e94153910 2024-01-20T08:59:52.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.945+0000 7f5e93fff640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 0x7f5e941e52c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400e9b0 tx=0x7f5e8400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:52.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.949+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84009670 con 0x7f5e94153910 2024-01-20T08:59:52.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.949+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5e8400c9a0 con 0x7f5e94153910 2024-01-20T08:59:52.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.949+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9415a9f0 con 0x7f5e94153910 2024-01-20T08:59:52.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.949+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400cd70 con 0x7f5e94153910 2024-01-20T08:59:52.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.949+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9414f4e0 con 0x7f5e94153910 2024-01-20T08:59:52.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.950+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e94067b60 con 0x7f5e94153910 2024-01-20T08:59:52.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.969+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5e84013470 con 0x7f5e94153910 2024-01-20T08:59:52.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.969+0000 7f5e90ff9640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5e7406aed0 0x7f5e7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:52.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.969+0000 7f5e937fe640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5e7406aed0 0x7f5e7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:52.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:52.969+0000 7f5e937fe640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5e7406aed0 0x7f5e7406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c036000 tx=0x7f5e7c008030 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:53.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.201+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f5e84058d50 con 0x7f5e94153910 2024-01-20T08:59:53.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.201+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5e84055000 con 0x7f5e94153910 2024-01-20T08:59:53.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.408+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e9404e9e0 con 0x7f5e94153910 2024-01-20T08:59:53.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.422+0000 7f5e90ff9640 1 -- 172.21.15.87:0/4194542324 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6584 (secure 0 0 0) 0x7f5e84021020 con 0x7f5e94153910 2024-01-20T08:59:53.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:53.424 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2838037730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2838037730},{"type":"v1","addr":"172.21.15.116:6835","nonce":2838037730}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-20T08:31:09.352915+0000","modified":"2024-01-20T08:31:45.461258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4290,"mds_2":4287,"mds_3":4284,"mds_4":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":4,"incarnation":17,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6837/1361375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1361375250},{"type":"v1","addr":"172.21.15.116:6837","nonce":1361375250}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":12,"addr":"172.21.15.87:6835/890269228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":890269228},{"type":"v1","addr":"172.21.15.87:6835","nonce":890269228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4287":{"gid":4287,"name":"c","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6837/279334988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":279334988},{"type":"v1","addr":"172.21.15.87:6837","nonce":279334988}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4290":{"gid":4290,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6833/1888537461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1888537461},{"type":"v1","addr":"172.21.15.116:6833","nonce":1888537461}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6839/3149977016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3149977016},{"type":"v1","addr":"172.21.15.87:6839","nonce":3149977016}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-20T08:59:53.424 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 18 2024-01-20T08:59:53.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5e7406aed0 msgr2=0x7f5e7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5e7406aed0 0x7f5e7406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c036000 tx=0x7f5e7c008030 comp rx=0 tx=0).stop 2024-01-20T08:59:53.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 msgr2=0x7f5e941e52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 0x7f5e941e52c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400e9b0 tx=0x7f5e8400ee70 comp rx=0 tx=0).stop 2024-01-20T08:59:53.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 shutdown_connections 2024-01-20T08:59:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f5e7406aed0 0x7f5e7406d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e9415d5c0 0x7f5e9415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e9415d000 0x7f5e9414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 --2- 172.21.15.87:0/4194542324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e94153910 0x7f5e941e52c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 >> 172.21.15.87:0/4194542324 conn(0x7f5e94071ce0 msgr2=0x7f5e941510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 shutdown_connections 2024-01-20T08:59:53.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.424+0000 7f5e9919e640 1 -- 172.21.15.87:0/4194542324 wait complete. 2024-01-20T08:59:53.436 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-20T08:59:53.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.535+0000 7f4f1e397640 1 Processor -- start 2024-01-20T08:59:53.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.535+0000 7f4f1e397640 1 -- start start 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f1e397640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4f18159770 0x7f4f1806af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f1e397640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1806fac0 con 0x7f4f180cc470 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f1e397640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1806fd60 con 0x7f4f180cc090 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f1e397640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1815d2c0 con 0x7f4f18159770 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f177fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4f18159770 0x7f4f1806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f17fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4f180cc090 0x7f4f1815d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:40358/0 (socket says 172.21.15.87:40358) 2024-01-20T08:59:53.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.536+0000 7f4f17fff640 1 -- 172.21.15.87:0/1979954692 learned_addr learned my addr 172.21.15.87:0/1979954692 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:53.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.539+0000 7f4f16ffd640 1 -- 172.21.15.87:0/1979954692 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966655020 0 0) 0x7f4f1806fd60 con 0x7f4f180cc090 2024-01-20T08:59:53.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.539+0000 7f4f16ffd640 1 -- 172.21.15.87:0/1979954692 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f00003520 con 0x7f4f180cc090 2024-01-20T08:59:53.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.541+0000 7f4f177fe640 1 -- 172.21.15.87:0/1979954692 >> v1:172.21.15.116:6789/0 conn(0x7f4f180cc090 legacy=0x7f4f1815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.541+0000 7f4f177fe640 1 -- 172.21.15.87:0/1979954692 >> v1:172.21.15.87:6789/0 conn(0x7f4f180cc470 legacy=0x7f4f181535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.541+0000 7f4f177fe640 1 -- 172.21.15.87:0/1979954692 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1815cee0 con 0x7f4f18159770 2024-01-20T08:59:53.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.543+0000 7f4f177fe640 1 --2- 172.21.15.87:0/1979954692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1806af00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0097a0 tx=0x7f4f0c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991a437bb2225e6f server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:53.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.548+0000 7f4f16ffd640 1 -- 172.21.15.87:0/1979954692 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c02f810 con 0x7f4f18159770 2024-01-20T08:59:53.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.548+0000 7f4f16ffd640 1 -- 172.21.15.87:0/1979954692 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4f0c02f970 con 0x7f4f18159770 2024-01-20T08:59:53.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.548+0000 7f4f16ffd640 1 -- 172.21.15.87:0/1979954692 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c02fd20 con 0x7f4f18159770 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 -- 172.21.15.87:0/1979954692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 msgr2=0x7f4f1806af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 --2- 172.21.15.87:0/1979954692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1806af00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0097a0 tx=0x7f4f0c02b9f0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 -- 172.21.15.87:0/1979954692 shutdown_connections 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 --2- 172.21.15.87:0/1979954692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1806af00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 -- 172.21.15.87:0/1979954692 >> 172.21.15.87:0/1979954692 conn(0x7f4f18071ff0 msgr2=0x7f4f18072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 -- 172.21.15.87:0/1979954692 shutdown_connections 2024-01-20T08:59:53.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.549+0000 7f4f1e397640 1 -- 172.21.15.87:0/1979954692 wait complete. 2024-01-20T08:59:53.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.550+0000 7f4f1e397640 1 Processor -- start 2024-01-20T08:59:53.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.550+0000 7f4f1e397640 1 -- start start 2024-01-20T08:59:53.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.551+0000 7f4f1e397640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f180cc090 0x7f4f180c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:53.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.551+0000 7f4f1e397640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f180cc470 0x7f4f1814de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:53.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.551+0000 7f4f1e397640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:53.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.551+0000 7f4f1e397640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180c55c0 con 0x7f4f18159770 2024-01-20T08:59:53.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.551+0000 7f4f1e397640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f181520f0 con 0x7f4f180cc470 2024-01-20T08:59:53.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.551+0000 7f4f1e397640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180c5830 con 0x7f4f180cc090 2024-01-20T08:59:53.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.552+0000 7f4f1c934640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:53.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.552+0000 7f4f177fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f180cc470 0x7f4f1814de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:53.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.552+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f180cc090 0x7f4f180c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:53.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.552+0000 7f4f1c934640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1814f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38422/0 (socket says 172.21.15.87:38422) 2024-01-20T08:59:53.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.552+0000 7f4f177fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f180cc470 0x7f4f1814de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:53488/0 (socket says 172.21.15.87:53488) 2024-01-20T08:59:53.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.552+0000 7f4f1c934640 1 -- 172.21.15.87:0/3154690274 learned_addr learned my addr 172.21.15.87:0/3154690274 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:53.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.555+0000 7f4f1c934640 1 -- 172.21.15.87:0/3154690274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f180cc090 msgr2=0x7f4f180c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.555+0000 7f4f1c934640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f180cc090 0x7f4f180c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.555+0000 7f4f1c934640 1 -- 172.21.15.87:0/3154690274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f180cc470 msgr2=0x7f4f1814de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:53.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.555+0000 7f4f1c934640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f180cc470 0x7f4f1814de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:53.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.555+0000 7f4f1c934640 1 -- 172.21.15.87:0/3154690274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009400 con 0x7f4f18159770 2024-01-20T08:59:53.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.558+0000 7f4f1c934640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1814f6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c760 tx=0x7f4f1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:53.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.561+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10003b70 con 0x7f4f18159770 2024-01-20T08:59:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.561+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4f10003cd0 con 0x7f4f18159770 2024-01-20T08:59:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.561+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000aab0 con 0x7f4f18159770 2024-01-20T08:59:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.561+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f181515b0 con 0x7f4f18159770 2024-01-20T08:59:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.562+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f18151850 con 0x7f4f18159770 2024-01-20T08:59:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.562+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f180c59e0 con 0x7f4f18159770 2024-01-20T08:59:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.804+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4f1000ac10 con 0x7f4f18159770 2024-01-20T08:59:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.804+0000 7f4f14ff9640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4f0406af20 0x7f4f0406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.804+0000 7f4f17fff640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4f0406af20 0x7f4f0406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:53.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.805+0000 7f4f17fff640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4f0406af20 0x7f4f0406d3e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005880 tx=0x7f4f08009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.819+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4f10058860 con 0x7f4f18159770 2024-01-20T08:59:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:53.819+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4f10054ba0 con 0x7f4f18159770 2024-01-20T08:59:54.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.002+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f180c59e0 con 0x7f4f18159770 2024-01-20T08:59:54.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.041+0000 7f4f14ff9640 1 -- 172.21.15.87:0/3154690274 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18446 (secure 0 0 0) 0x7f4f180c59e0 con 0x7f4f18159770 2024-01-20T08:59:54.043 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-01-20T08:59:54.043 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"fsid":"2b2378c9-6207-4be4-af26-3b0cbbc1772f","created":"2024-01-20T08:30:31.879489+0000","modified":"2024-01-20T08:31:43.948324+0000","last_up_change":"2024-01-20T08:30:46.660902+0000","last_in_change":"2024-01-20T08:30:37.131717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-20T08:30:44.696145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-20T08:31:06.500533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-20T08:31:08.175669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-20T08:31:13.903374+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"59b49896-16a3-4471-834a-3f9977004332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6801","nonce":1449648137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6803","nonce":1449648137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6807","nonce":1449648137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1449648137},{"type":"v1","addr":"172.21.15.87:6805","nonce":1449648137}]},"public_addr":"172.21.15.87:6801/1449648137","cluster_addr":"172.21.15.87:6803/1449648137","heartbeat_back_addr":"172.21.15.87:6807/1449648137","heartbeat_front_addr":"172.21.15.87:6805/1449648137","state":["exists","up"]},{"osd":1,"uuid":"bac24976-24c1-425c-988e-ace7d4640e3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6821","nonce":2572242901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6823","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6825","nonce":2572242901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6831","nonce":2572242901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2572242901},{"type":"v1","addr":"172.21.15.87:6829","nonce":2572242901}]},"public_addr":"172.21.15.87:6821/2572242901","cluster_addr":"172.21.15.87:6825/2572242901","heartbeat_back_addr":"172.21.15.87:6831/2572242901","heartbeat_front_addr":"172.21.15.87:6829/2572242901","state":["exists","up"]},{"osd":2,"uuid":"28b3c7c6-3c5e-4309-a825-0945264488ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6809","nonce":3102513365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6811","nonce":3102513365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6815","nonce":3102513365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3102513365},{"type":"v1","addr":"172.21.15.87:6813","nonce":3102513365}]},"public_addr":"172.21.15.87:6809/3102513365","cluster_addr":"172.21.15.87:6811/3102513365","heartbeat_back_addr":"172.21.15.87:6815/3102513365","heartbeat_front_addr":"172.21.15.87:6813/3102513365","state":["exists","up"]},{"osd":3,"uuid":"3a06fe92-a35d-439c-a9be-ed5afb82c6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6817","nonce":973971716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6819","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6820","nonce":973971716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6827","nonce":973971716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":973971716},{"type":"v1","addr":"172.21.15.87:6824","nonce":973971716}]},"public_addr":"172.21.15.87:6817/973971716","cluster_addr":"172.21.15.87:6820/973971716","heartbeat_back_addr":"172.21.15.87:6827/973971716","heartbeat_front_addr":"172.21.15.87:6824/973971716","state":["exists","up"]},{"osd":4,"uuid":"d6386370-b971-4716-a270-0192527c4863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6809","nonce":2305241895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6811","nonce":2305241895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6815","nonce":2305241895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2305241895},{"type":"v1","addr":"172.21.15.116:6813","nonce":2305241895}]},"public_addr":"172.21.15.116:6809/2305241895","cluster_addr":"172.21.15.116:6811/2305241895","heartbeat_back_addr":"172.21.15.116:6815/2305241895","heartbeat_front_addr":"172.21.15.116:6813/2305241895","state":["exists","up"]},{"osd":5,"uuid":"c78a9699-c969-4c6a-8cc6-3159de537dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6817","nonce":4162650663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6819","nonce":4162650663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6826","nonce":4162650663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":4162650663},{"type":"v1","addr":"172.21.15.116:6822","nonce":4162650663}]},"public_addr":"172.21.15.116:6817/4162650663","cluster_addr":"172.21.15.116:6819/4162650663","heartbeat_back_addr":"172.21.15.116:6826/4162650663","heartbeat_front_addr":"172.21.15.116:6822/4162650663","state":["exists","up"]},{"osd":6,"uuid":"389dcf96-e639-45ee-b53c-0efd01b1be3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6823","nonce":3458193509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6827","nonce":3458193509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6831","nonce":3458193509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3458193509},{"type":"v1","addr":"172.21.15.116:6829","nonce":3458193509}]},"public_addr":"172.21.15.116:6823/3458193509","cluster_addr":"172.21.15.116:6827/3458193509","heartbeat_back_addr":"172.21.15.116:6831/3458193509","heartbeat_front_addr":"172.21.15.116:6829/3458193509","state":["exists","up"]},{"osd":7,"uuid":"98176cde-6f9f-43be-96db-561e583ffacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6801","nonce":1436327026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6803","nonce":1436327026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6807","nonce":1436327026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1436327026},{"type":"v1","addr":"172.21.15.116:6805","nonce":1436327026}]},"public_addr":"172.21.15.116:6801/1436327026","cluster_addr":"172.21.15.116:6803/1436327026","heartbeat_back_addr":"172.21.15.116:6807/1436327026","heartbeat_front_addr":"172.21.15.116:6805/1436327026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.325904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.332639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.350858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.835777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.363077+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.391027+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.397615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-20T08:30:42.852215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]},{"pgid":"4.19","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.043+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4f0406af20 msgr2=0x7f4f0406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.043+0000 7f4f1e397640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4f0406af20 0x7f4f0406d3e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005880 tx=0x7f4f08009040 comp rx=0 tx=0).stop 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.043+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 msgr2=0x7f4f1814f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.043+0000 7f4f1e397640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1814f6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c760 tx=0x7f4f1000cc20 comp rx=0 tx=0).stop 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 shutdown_connections 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f4f0406af20 0x7f4f0406d3e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f18159770 0x7f4f1814f6d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f180cc470 0x7f4f1814de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 --2- 172.21.15.87:0/3154690274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f180cc090 0x7f4f180c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 >> 172.21.15.87:0/3154690274 conn(0x7f4f18071ff0 msgr2=0x7f4f180c7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:54.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 shutdown_connections 2024-01-20T08:59:54.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.044+0000 7f4f1e397640 1 -- 172.21.15.87:0/3154690274 wait complete. 2024-01-20T08:59:54.056 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-20T08:59:54.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.156+0000 7f7d9e5fe640 1 Processor -- start 2024-01-20T08:59:54.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.157+0000 7f7d9e5fe640 1 -- start start 2024-01-20T08:59:54.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.157+0000 7f7d9e5fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d98153bb0 0x7f7d980c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.157+0000 7f7d9e5fe640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d980c7920 con 0x7f7d980c21a0 2024-01-20T08:59:54.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.157+0000 7f7d9e5fe640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d981597f0 con 0x7f7d980c0e80 2024-01-20T08:59:54.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.157+0000 7f7d9e5fe640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980c6ca0 con 0x7f7d98153bb0 2024-01-20T08:59:54.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.158+0000 7f7d9cb9b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7d980c21a0 0x7f7d98159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51422/0 (socket says 172.21.15.87:51422) 2024-01-20T08:59:54.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.158+0000 7f7d9cb9b640 1 -- 172.21.15.87:0/1932539892 learned_addr learned my addr 172.21.15.87:0/1932539892 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:54.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.158+0000 7f7d97fff640 1 --2- 172.21.15.87:0/1932539892 >> v2:172.21.15.116:3300/0 conn(0x7f7d98153bb0 0x7f7d980c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.160+0000 7f7d96ffd640 1 -- 172.21.15.87:0/1932539892 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636629801 0 0) 0x7f7d980c7920 con 0x7f7d980c21a0 2024-01-20T08:59:54.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.160+0000 7f7d96ffd640 1 -- 172.21.15.87:0/1932539892 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d80003520 con 0x7f7d980c21a0 2024-01-20T08:59:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.162+0000 7f7d97fff640 1 -- 172.21.15.87:0/1932539892 >> v1:172.21.15.116:6789/0 conn(0x7f7d980c0e80 legacy=0x7f7d980c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.162+0000 7f7d97fff640 1 -- 172.21.15.87:0/1932539892 >> v1:172.21.15.87:6789/0 conn(0x7f7d980c21a0 legacy=0x7f7d98159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.162+0000 7f7d97fff640 1 -- 172.21.15.87:0/1932539892 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d98153f90 con 0x7f7d98153bb0 2024-01-20T08:59:54.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.165+0000 7f7d97fff640 1 --2- 172.21.15.87:0/1932539892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d98153bb0 0x7f7d980c6760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d880097a0 tx=0x7f7d8802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40768dd94cdc5472 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:54.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.168+0000 7f7d96ffd640 1 -- 172.21.15.87:0/1932539892 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88030610 con 0x7f7d98153bb0 2024-01-20T08:59:54.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d96ffd640 1 -- 172.21.15.87:0/1932539892 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7d88030770 con 0x7f7d98153bb0 2024-01-20T08:59:54.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d96ffd640 1 -- 172.21.15.87:0/1932539892 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88030b20 con 0x7f7d98153bb0 2024-01-20T08:59:54.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/1932539892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d98153bb0 msgr2=0x7f7d980c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/1932539892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d98153bb0 0x7f7d980c6760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d880097a0 tx=0x7f7d8802b960 comp rx=0 tx=0).stop 2024-01-20T08:59:54.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/1932539892 shutdown_connections 2024-01-20T08:59:54.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/1932539892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d98153bb0 0x7f7d980c6760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/1932539892 >> 172.21.15.87:0/1932539892 conn(0x7f7d980723c0 msgr2=0x7f7d9806fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:54.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/1932539892 shutdown_connections 2024-01-20T08:59:54.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.169+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/1932539892 wait complete. 2024-01-20T08:59:54.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.170+0000 7f7d9e5fe640 1 Processor -- start 2024-01-20T08:59:54.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 -- start start 2024-01-20T08:59:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d980c0e80 0x7f7d98152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 0x7f7d9814b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7d98153bb0 0x7f7d9814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980aca50 con 0x7f7d980c21a0 2024-01-20T08:59:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980adc70 con 0x7f7d980c0e80 2024-01-20T08:59:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.171+0000 7f7d9e5fe640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9814e930 con 0x7f7d98153bb0 2024-01-20T08:59:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.172+0000 7f7d977fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 0x7f7d9814b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.172+0000 7f7d97fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d980c0e80 0x7f7d98152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.172+0000 7f7d9cb9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7d98153bb0 0x7f7d9814bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.172+0000 7f7d977fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 0x7f7d9814b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38424/0 (socket says 172.21.15.87:38424) 2024-01-20T08:59:54.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.172+0000 7f7d977fe640 1 -- 172.21.15.87:0/261948975 learned_addr learned my addr 172.21.15.87:0/261948975 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:54.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.175+0000 7f7d977fe640 1 -- 172.21.15.87:0/261948975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7d98153bb0 msgr2=0x7f7d9814bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.175+0000 7f7d977fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7d98153bb0 0x7f7d9814bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.175+0000 7f7d977fe640 1 -- 172.21.15.87:0/261948975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d980c0e80 msgr2=0x7f7d98152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.175+0000 7f7d977fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d980c0e80 0x7f7d98152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.175+0000 7f7d977fe640 1 -- 172.21.15.87:0/261948975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88009400 con 0x7f7d980c21a0 2024-01-20T08:59:54.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.178+0000 7f7d977fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 0x7f7d9814b600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c400 tx=0x7f7d8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.181+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0052d0 con 0x7f7d980c21a0 2024-01-20T08:59:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.181+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7d8c005450 con 0x7f7d980c21a0 2024-01-20T08:59:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.181+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9814d200 con 0x7f7d980c21a0 2024-01-20T08:59:54.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.181+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c007280 con 0x7f7d980c21a0 2024-01-20T08:59:54.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.181+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9814d7c0 con 0x7f7d980c21a0 2024-01-20T08:59:54.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.182+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9814eb30 con 0x7f7d980c21a0 2024-01-20T08:59:54.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.199+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7d8c0073e0 con 0x7f7d980c21a0 2024-01-20T08:59:54.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.200+0000 7f7d94ff9640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7d8406af50 0x7f7d8406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.200+0000 7f7d97fff640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7d8406af50 0x7f7d8406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.200+0000 7f7d97fff640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7d8406af50 0x7f7d8406d410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d88036000 tx=0x7f7d8803b040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:54.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.203+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f7d8c057cc0 con 0x7f7d980c21a0 2024-01-20T08:59:54.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.437+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7d8c054000 con 0x7f7d980c21a0 2024-01-20T08:59:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.663+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7d9814eaf0 con 0x7f7d980c21a0 2024-01-20T08:59:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.747+0000 7f7d94ff9640 1 -- 172.21.15.87:0/261948975 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7d8c020020 con 0x7f7d980c21a0 2024-01-20T08:59:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.749+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7d8406af50 msgr2=0x7f7d8406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.749+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7d8406af50 0x7f7d8406d410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d88036000 tx=0x7f7d8803b040 comp rx=0 tx=0).stop 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.749+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 msgr2=0x7f7d9814b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.749+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 0x7f7d9814b600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c400 tx=0x7f7d8c00c8c0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 shutdown_connections 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7d8406af50 0x7f7d8406d410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7d98153bb0 0x7f7d9814bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d980c21a0 0x7f7d9814b600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 --2- 172.21.15.87:0/261948975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d980c0e80 0x7f7d98152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 >> 172.21.15.87:0/261948975 conn(0x7f7d980723c0 msgr2=0x7f7d98158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:54.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 shutdown_connections 2024-01-20T08:59:54.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.750+0000 7f7d9e5fe640 1 -- 172.21.15.87:0/261948975 wait complete. 2024-01-20T08:59:54.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-20T08:59:54.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.872+0000 7f954a0da640 1 Processor -- start 2024-01-20T08:59:54.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.872+0000 7f954a0da640 1 -- start start 2024-01-20T08:59:54.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.873+0000 7f954a0da640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f95440c9e00 0x7f954415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.873+0000 7f954a0da640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954415a7f0 con 0x7f9544153a20 2024-01-20T08:59:54.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.873+0000 7f954a0da640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95440cc3c0 con 0x7f95440cefd0 2024-01-20T08:59:54.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.873+0000 7f954a0da640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95440ca780 con 0x7f95440c9e00 2024-01-20T08:59:54.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.874+0000 7f9543fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f95440cefd0 0x7f9544164110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.87:40364/0 (socket says 172.21.15.87:40364) 2024-01-20T08:59:54.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.874+0000 7f9543fff640 1 -- 172.21.15.87:0/1816931247 learned_addr learned my addr 172.21.15.87:0/1816931247 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-01-20T08:59:54.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.874+0000 7f9542ffd640 1 --2- 172.21.15.87:0/1816931247 >> v2:172.21.15.87:3300/0 conn(0x7f95440c9e00 0x7f954415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.878+0000 7f95427fc640 1 -- 172.21.15.87:0/1816931247 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397477969 0 0) 0x7f95440cc3c0 con 0x7f95440cefd0 2024-01-20T08:59:54.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.878+0000 7f95427fc640 1 -- 172.21.15.87:0/1816931247 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f952c003520 con 0x7f95440cefd0 2024-01-20T08:59:54.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.879+0000 7f9542ffd640 1 -- 172.21.15.87:0/1816931247 >> v1:172.21.15.116:6789/0 conn(0x7f95440cefd0 legacy=0x7f9544164110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.879+0000 7f9542ffd640 1 -- 172.21.15.87:0/1816931247 >> v1:172.21.15.87:6789/0 conn(0x7f9544153a20 legacy=0x7f95440c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.879+0000 7f9542ffd640 1 -- 172.21.15.87:0/1816931247 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440c4670 con 0x7f95440c9e00 2024-01-20T08:59:54.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.884+0000 7f9542ffd640 1 --2- 172.21.15.87:0/1816931247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95440c9e00 0x7f954415a2b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95380097a0 tx=0x7f953802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa26d4b90e3005c server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:54.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.889+0000 7f95427fc640 1 -- 172.21.15.87:0/1816931247 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538031830 con 0x7f95440c9e00 2024-01-20T08:59:54.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.889+0000 7f95427fc640 1 -- 172.21.15.87:0/1816931247 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9538031990 con 0x7f95440c9e00 2024-01-20T08:59:54.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.889+0000 7f95427fc640 1 -- 172.21.15.87:0/1816931247 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538031d40 con 0x7f95440c9e00 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.889+0000 7f954a0da640 1 -- 172.21.15.87:0/1816931247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95440c9e00 msgr2=0x7f954415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.889+0000 7f954a0da640 1 --2- 172.21.15.87:0/1816931247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95440c9e00 0x7f954415a2b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95380097a0 tx=0x7f953802b8d0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.890+0000 7f954a0da640 1 -- 172.21.15.87:0/1816931247 shutdown_connections 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.890+0000 7f954a0da640 1 --2- 172.21.15.87:0/1816931247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95440c9e00 0x7f954415a2b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.890+0000 7f954a0da640 1 -- 172.21.15.87:0/1816931247 >> 172.21.15.87:0/1816931247 conn(0x7f954408af90 msgr2=0x7f954408b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.890+0000 7f954a0da640 1 -- 172.21.15.87:0/1816931247 shutdown_connections 2024-01-20T08:59:54.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.890+0000 7f954a0da640 1 -- 172.21.15.87:0/1816931247 wait complete. 2024-01-20T08:59:54.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.891+0000 7f954a0da640 1 Processor -- start 2024-01-20T08:59:54.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.891+0000 7f954a0da640 1 -- start start 2024-01-20T08:59:54.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.892+0000 7f954a0da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95440c9e00 0x7f9544145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.892+0000 7f954a0da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95440cefd0 0x7f95440b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.892+0000 7f954a0da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9544153a20 0x7f95441ef990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.892+0000 7f954a0da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954414e880 con 0x7f9544153a20 2024-01-20T08:59:54.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.892+0000 7f954a0da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441ef500 con 0x7f95440cefd0 2024-01-20T08:59:54.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.892+0000 7f954a0da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954414eaa0 con 0x7f95440c9e00 2024-01-20T08:59:54.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.893+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95440cefd0 0x7f95440b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.893+0000 7f95437fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95440c9e00 0x7f9544145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.893+0000 7f9543fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9544153a20 0x7f95441ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.893+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95440cefd0 0x7f95440b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.87:53514/0 (socket says 172.21.15.87:53514) 2024-01-20T08:59:54.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.893+0000 7f9542ffd640 1 -- 172.21.15.87:0/736268653 learned_addr learned my addr 172.21.15.87:0/736268653 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.896+0000 7f9543fff640 1 -- 172.21.15.87:0/736268653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95440c9e00 msgr2=0x7f9544145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.896+0000 7f9543fff640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95440c9e00 0x7f9544145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.896+0000 7f9543fff640 1 -- 172.21.15.87:0/736268653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95440cefd0 msgr2=0x7f95440b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.896+0000 7f9543fff640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95440cefd0 0x7f95440b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.896+0000 7f9543fff640 1 -- 172.21.15.87:0/736268653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538009400 con 0x7f9544153a20 2024-01-20T08:59:54.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.899+0000 7f9543fff640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9544153a20 0x7f95441ef990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f953c00ca80 tx=0x7f953c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:54.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.903+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00ac30 con 0x7f9544153a20 2024-01-20T08:59:54.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.903+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f953c00ad90 con 0x7f9544153a20 2024-01-20T08:59:54.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.903+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c012620 con 0x7f9544153a20 2024-01-20T08:59:54.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.903+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954414ed80 con 0x7f9544153a20 2024-01-20T08:59:54.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.903+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95441ef190 con 0x7f9544153a20 2024-01-20T08:59:54.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.904+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95441ef320 con 0x7f9544153a20 2024-01-20T08:59:54.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.927+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f953c007070 con 0x7f9544153a20 2024-01-20T08:59:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.928+0000 7f952bfff640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f953006aed0 0x7f953006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.928+0000 7f95437fe640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f953006aed0 0x7f953006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.928+0000 7f95437fe640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f953006aed0 0x7f953006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95340016e0 tx=0x7f9534009040 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:54.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:54.931+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f953c058620 con 0x7f9544153a20 2024-01-20T08:59:55.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:55.276+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f953c054960 con 0x7f9544153a20 2024-01-20T08:59:55.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:55.621+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9544074290 con 0x7f9544153a20 2024-01-20T08:59:55.908 INFO:tasks.ceph.mds.d.smithi116.stderr:==00:00:29:01.902 41990== 2024-01-20T08:59:55.908 INFO:tasks.ceph.mds.d.smithi116.stderr:==00:00:29:01.902 41990== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T08:59:55.945 INFO:tasks.ceph.mds.f.smithi116.stderr:==00:00:29:01.937 41991== 2024-01-20T08:59:55.945 INFO:tasks.ceph.mds.f.smithi116.stderr:==00:00:29:01.937 41991== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T08:59:55.995 INFO:tasks.ceph.mds.f.smithi116.stderr:daemon-helper: command failed with exit status 42 2024-01-20T08:59:55.995 INFO:tasks.ceph.mds.d.smithi116.stderr:daemon-helper: command failed with exit status 42 2024-01-20T08:59:56.050 INFO:tasks.ceph.mds.a.smithi087.stderr:==00:00:29:02.067 41688== 2024-01-20T08:59:56.050 INFO:tasks.ceph.mds.a.smithi087.stderr:==00:00:29:02.067 41688== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T08:59:56.057 INFO:tasks.ceph.mds.e.smithi087.stderr:==00:00:29:02.072 41690== 2024-01-20T08:59:56.057 INFO:tasks.ceph.mds.e.smithi087.stderr:==00:00:29:02.072 41690== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T08:59:56.111 INFO:tasks.ceph.mds.c.smithi087.stderr:==00:00:29:02.128 41689== 2024-01-20T08:59:56.112 INFO:tasks.ceph.mds.c.smithi087.stderr:==00:00:29:02.128 41689== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T08:59:56.234 INFO:tasks.ceph.mds.a.smithi087.stderr:daemon-helper: command failed with exit status 42 2024-01-20T08:59:56.234 INFO:tasks.ceph.mds.c.smithi087.stderr:daemon-helper: command failed with exit status 42 2024-01-20T08:59:56.234 INFO:tasks.ceph.mds.e.smithi087.stderr:daemon-helper: command failed with exit status 42 2024-01-20T08:59:56.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.585+0000 7f952bfff640 1 -- 172.21.15.87:0/736268653 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f953c017070 con 0x7f9544153a20 2024-01-20T08:59:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f953006aed0 msgr2=0x7f953006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f953006aed0 0x7f953006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95340016e0 tx=0x7f9534009040 comp rx=0 tx=0).stop 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9544153a20 msgr2=0x7f95441ef990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9544153a20 0x7f95441ef990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f953c00ca80 tx=0x7f953c00cf40 comp rx=0 tx=0).stop 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 shutdown_connections 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f953006aed0 0x7f953006d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9544153a20 0x7f95441ef990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95440cefd0 0x7f95440b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 --2- 172.21.15.87:0/736268653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95440c9e00 0x7f9544145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 >> 172.21.15.87:0/736268653 conn(0x7f954408af90 msgr2=0x7f954408b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 shutdown_connections 2024-01-20T08:59:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.587+0000 7f954a0da640 1 -- 172.21.15.87:0/736268653 wait complete. 2024-01-20T08:59:56.600 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-20T08:59:56.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.705+0000 7f098a3cb640 1 Processor -- start 2024-01-20T08:59:56.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.705+0000 7f098a3cb640 1 -- start start 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f098a3cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f098406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f098a3cb640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0984159770 0x7f09840c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f098a3cb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09840cc470 0x7f098415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f098a3cb640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098415d2c0 con 0x7f0984159770 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f098a3cb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09840c6e10 con 0x7f09840cc470 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f098a3cb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098406b540 con 0x7f09840cc090 2024-01-20T08:59:56.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.706+0000 7f09837fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0984159770 0x7f09840c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.707+0000 7f0983fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f098406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.707+0000 7f09837fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0984159770 0x7f09840c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53956/0 (socket says 172.21.15.87:53956) 2024-01-20T08:59:56.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.707+0000 7f09837fe640 1 -- 172.21.15.87:0/2903015510 learned_addr learned my addr 172.21.15.87:0/2903015510 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:56.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.707+0000 7f0988968640 1 --2- 172.21.15.87:0/2903015510 >> v2:172.21.15.116:3300/0 conn(0x7f09840cc470 0x7f098415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.709+0000 7f09837fe640 1 -- 172.21.15.87:0/2903015510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 msgr2=0x7f098406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.709+0000 7f09837fe640 1 --2- 172.21.15.87:0/2903015510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f098406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.709+0000 7f09837fe640 1 -- 172.21.15.87:0/2903015510 >> v2:172.21.15.116:3300/0 conn(0x7f09840cc470 msgr2=0x7f098415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.709+0000 7f09837fe640 1 --2- 172.21.15.87:0/2903015510 >> v2:172.21.15.116:3300/0 conn(0x7f09840cc470 0x7f098415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.709+0000 7f09837fe640 1 -- 172.21.15.87:0/2903015510 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098406fac0 con 0x7f0984159770 2024-01-20T08:59:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.711+0000 7f09837fe640 1 --2- 172.21.15.87:0/2903015510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0984159770 0x7f09840c68d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09840702f0 tx=0x7f096c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1fd919f94dbe06 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.713+0000 7f0982ffd640 1 -- 172.21.15.87:0/2903015510 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c030860 con 0x7f0984159770 2024-01-20T08:59:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.714+0000 7f0982ffd640 1 -- 172.21.15.87:0/2903015510 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f096c0309c0 con 0x7f0984159770 2024-01-20T08:59:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.714+0000 7f0982ffd640 1 -- 172.21.15.87:0/2903015510 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c030d70 con 0x7f0984159770 2024-01-20T08:59:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 -- 172.21.15.87:0/2903015510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0984159770 msgr2=0x7f09840c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 --2- 172.21.15.87:0/2903015510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0984159770 0x7f09840c68d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09840702f0 tx=0x7f096c02b9a0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 -- 172.21.15.87:0/2903015510 shutdown_connections 2024-01-20T08:59:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 --2- 172.21.15.87:0/2903015510 >> v2:172.21.15.116:3300/0 conn(0x7f09840cc470 0x7f098415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 --2- 172.21.15.87:0/2903015510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0984159770 0x7f09840c68d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 --2- 172.21.15.87:0/2903015510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f098406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 -- 172.21.15.87:0/2903015510 >> 172.21.15.87:0/2903015510 conn(0x7f0984071ff0 msgr2=0x7f0984072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 -- 172.21.15.87:0/2903015510 shutdown_connections 2024-01-20T08:59:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.715+0000 7f098a3cb640 1 -- 172.21.15.87:0/2903015510 wait complete. 2024-01-20T08:59:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.716+0000 7f098a3cb640 1 Processor -- start 2024-01-20T08:59:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 -- start start 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f0984157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 0x7f09841f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984159770 0x7f09840c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0984155400 con 0x7f09840cc470 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0984154b60 con 0x7f0984159770 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.717+0000 7f098a3cb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0984155f70 con 0x7f09840cc090 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.718+0000 7f09837fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 0x7f09841f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.718+0000 7f0983fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f0984157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.718+0000 7f0988968640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984159770 0x7f09840c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.718+0000 7f09837fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 0x7f09841f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53966/0 (socket says 172.21.15.87:53966) 2024-01-20T08:59:56.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.718+0000 7f09837fe640 1 -- 172.21.15.87:0/1082144372 learned_addr learned my addr 172.21.15.87:0/1082144372 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:56.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.721+0000 7f09837fe640 1 -- 172.21.15.87:0/1082144372 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 msgr2=0x7f0984157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:56.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.721+0000 7f09837fe640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f0984157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.721+0000 7f09837fe640 1 -- 172.21.15.87:0/1082144372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984159770 msgr2=0x7f09840c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:56.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.721+0000 7f09837fe640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984159770 0x7f09840c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:56.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.721+0000 7f09837fe640 1 -- 172.21.15.87:0/1082144372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c0094d0 con 0x7f09840cc470 2024-01-20T08:59:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.724+0000 7f09837fe640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 0x7f09841f27c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f096c02b9e0 tx=0x7f096c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:56.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.727+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c039a90 con 0x7f09840cc470 2024-01-20T08:59:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.727+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f096c039bf0 con 0x7f09840cc470 2024-01-20T08:59:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.727+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0984156230 con 0x7f09840cc470 2024-01-20T08:59:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.727+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c043410 con 0x7f09840cc470 2024-01-20T08:59:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.728+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0984155090 con 0x7f09840cc470 2024-01-20T08:59:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.729+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098415cea0 con 0x7f09840cc470 2024-01-20T08:59:56.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.747+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f096c04c3b0 con 0x7f09840cc470 2024-01-20T08:59:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.747+0000 7f0980ff9640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f096406af20 0x7f096406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:56.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.748+0000 7f0983fff640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f096406af20 0x7f096406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:56.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.748+0000 7f0983fff640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f096406af20 0x7f096406d3e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0974009ac0 tx=0x7f0974009190 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:56.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.751+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f096c07acb0 con 0x7f09840cc470 2024-01-20T08:59:56.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-20T08:59:56.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-20T08:59:56.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2024-01-20T08:59:56.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-01-20T08:59:56.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2024-01-20T08:59:56.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:56.982+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f096c076ff0 con 0x7f09840cc470 2024-01-20T08:59:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.152+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0984155220 con 0x7f09840cc470 2024-01-20T08:59:57.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.679+0000 7f0980ff9640 1 -- 172.21.15.87:0/1082144372 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f096c0340d0 con 0x7f09840cc470 2024-01-20T08:59:57.680 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.681+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f096406af20 msgr2=0x7f096406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.681+0000 7f098a3cb640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f096406af20 0x7f096406d3e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0974009ac0 tx=0x7f0974009190 comp rx=0 tx=0).stop 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.681+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 msgr2=0x7f09841f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.681+0000 7f098a3cb640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 0x7f09841f27c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f096c02b9e0 tx=0x7f096c004730 comp rx=0 tx=0).stop 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 shutdown_connections 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f096406af20 0x7f096406d3e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984159770 0x7f09840c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09840cc470 0x7f09841f27c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 --2- 172.21.15.87:0/1082144372 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09840cc090 0x7f0984157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 >> 172.21.15.87:0/1082144372 conn(0x7f0984071ff0 msgr2=0x7f0984158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:57.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 shutdown_connections 2024-01-20T08:59:57.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.682+0000 7f098a3cb640 1 -- 172.21.15.87:0/1082144372 wait complete. 2024-01-20T08:59:57.700 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-20T08:59:57.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.811+0000 7f0c65f8d640 1 Processor -- start 2024-01-20T08:59:57.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.811+0000 7f0c65f8d640 1 -- start start 2024-01-20T08:59:57.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.812+0000 7f0c65f8d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c60153920 0x7f0c6006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:57.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.812+0000 7f0c65f8d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c6006dd10 0x7f0c60071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:57.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.812+0000 7f0c65f8d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6015da70 con 0x7f0c60071520 2024-01-20T08:59:57.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.812+0000 7f0c65f8d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c600b0660 con 0x7f0c6006dd10 2024-01-20T08:59:57.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.812+0000 7f0c65f8d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6006d1b0 con 0x7f0c60153920 2024-01-20T08:59:57.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.813+0000 7f0c5ffff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c6006dd10 0x7f0c60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:57.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.813+0000 7f0c5f7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c60153920 0x7f0c6006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:57.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.813+0000 7f0c5ffff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c6006dd10 0x7f0c60071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53982/0 (socket says 172.21.15.87:53982) 2024-01-20T08:59:57.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.813+0000 7f0c5ffff640 1 -- 172.21.15.87:0/3917836449 learned_addr learned my addr 172.21.15.87:0/3917836449 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:57.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.815+0000 7f0c5f7fe640 1 -- 172.21.15.87:0/3917836449 >> v2:172.21.15.87:3300/0 conn(0x7f0c6006dd10 msgr2=0x7f0c60071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:57.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.815+0000 7f0c5f7fe640 1 --2- 172.21.15.87:0/3917836449 >> v2:172.21.15.87:3300/0 conn(0x7f0c6006dd10 0x7f0c60071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.815+0000 7f0c5f7fe640 1 -- 172.21.15.87:0/3917836449 >> v1:172.21.15.87:6789/0 conn(0x7f0c60071520 legacy=0x7f0c6006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:57.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.815+0000 7f0c5f7fe640 1 -- 172.21.15.87:0/3917836449 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6015cf10 con 0x7f0c60153920 2024-01-20T08:59:57.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.817+0000 7f0c5f7fe640 1 --2- 172.21.15.87:0/3917836449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c60153920 0x7f0c6006f550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c600c56d0 tx=0x7f0c5002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bddde3f4edcd13c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:57.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c5e7fc640 1 -- 172.21.15.87:0/3917836449 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5002e070 con 0x7f0c60153920 2024-01-20T08:59:57.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c5e7fc640 1 -- 172.21.15.87:0/3917836449 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0c500315a0 con 0x7f0c60153920 2024-01-20T08:59:57.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c5e7fc640 1 -- 172.21.15.87:0/3917836449 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50031950 con 0x7f0c60153920 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3917836449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c60153920 msgr2=0x7f0c6006f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3917836449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c60153920 0x7f0c6006f550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c600c56d0 tx=0x7f0c5002b840 comp rx=0 tx=0).stop 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3917836449 shutdown_connections 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3917836449 >> v2:172.21.15.87:3300/0 conn(0x7f0c6006dd10 0x7f0c60071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3917836449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c60153920 0x7f0c6006f550 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.821+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3917836449 >> 172.21.15.87:0/3917836449 conn(0x7f0c6006abe0 msgr2=0x7f0c6006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.822+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3917836449 shutdown_connections 2024-01-20T08:59:57.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.822+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3917836449 wait complete. 2024-01-20T08:59:57.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 Processor -- start 2024-01-20T08:59:57.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 -- start start 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6006dd10 0x7f0c600c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c60071520 0x7f0c60158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c60153920 0x7f0c60158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60154730 con 0x7f0c60071520 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60158480 con 0x7f0c6006dd10 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.823+0000 7f0c65f8d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600c3c50 con 0x7f0c60153920 2024-01-20T08:59:57.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.824+0000 7f0c5effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c60071520 0x7f0c60158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:57.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.824+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c60153920 0x7f0c60158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:57.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.824+0000 7f0c5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6006dd10 0x7f0c600c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:57.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.824+0000 7f0c5effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c60071520 0x7f0c60158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53996/0 (socket says 172.21.15.87:53996) 2024-01-20T08:59:57.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.824+0000 7f0c5effd640 1 -- 172.21.15.87:0/3945061351 learned_addr learned my addr 172.21.15.87:0/3945061351 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:57.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.828+0000 7f0c5f7fe640 1 -- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c60153920 msgr2=0x7f0c60158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:57.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.828+0000 7f0c5f7fe640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c60153920 0x7f0c60158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.828+0000 7f0c5f7fe640 1 -- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c60071520 msgr2=0x7f0c60158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:57.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.828+0000 7f0c5f7fe640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c60071520 0x7f0c60158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:57.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.828+0000 7f0c5f7fe640 1 -- 172.21.15.87:0/3945061351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50009400 con 0x7f0c6006dd10 2024-01-20T08:59:57.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.830+0000 7f0c5f7fe640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6006dd10 0x7f0c600c0eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c50033000 tx=0x7f0c500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.834+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50032840 con 0x7f0c6006dd10 2024-01-20T08:59:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.834+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0c500329a0 con 0x7f0c6006dd10 2024-01-20T08:59:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.834+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50032d70 con 0x7f0c6006dd10 2024-01-20T08:59:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.834+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c600c3f10 con 0x7f0c6006dd10 2024-01-20T08:59:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.834+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c600c44d0 con 0x7f0c6006dd10 2024-01-20T08:59:57.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:57.835+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c60067260 con 0x7f0c6006dd10 2024-01-20T08:59:58.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.078+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0c5004e400 con 0x7f0c6006dd10 2024-01-20T08:59:58.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.078+0000 7f0c47fff640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c4c06af20 0x7f0c4c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.078+0000 7f0c5effd640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c4c06af20 0x7f0c4c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.079+0000 7f0c5effd640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c4c06af20 0x7f0c4c06d3e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c6015a1f0 tx=0x7f0c54007ef0 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.082+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6510+0+0 (secure 0 0 0) 0x7f0c5007a210 con 0x7f0c6006dd10 2024-01-20T08:59:58.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.092+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0c50076550 con 0x7f0c6006dd10 2024-01-20T08:59:58.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.265+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c60074290 con 0x7f0c6006dd10 2024-01-20T08:59:58.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.779+0000 7f0c47fff640 1 -- 172.21.15.87:0/3945061351 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0c50079b70 con 0x7f0c6006dd10 2024-01-20T08:59:58.781 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.781+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c4c06af20 msgr2=0x7f0c4c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.781+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c4c06af20 0x7f0c4c06d3e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c6015a1f0 tx=0x7f0c54007ef0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.781+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6006dd10 msgr2=0x7f0c600c0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.781+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6006dd10 0x7f0c600c0eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c50033000 tx=0x7f0c500046c0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 shutdown_connections 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f0c4c06af20 0x7f0c4c06d3e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0c60153920 0x7f0c60158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c60071520 0x7f0c60158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 --2- 172.21.15.87:0/3945061351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6006dd10 0x7f0c600c0eb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 >> 172.21.15.87:0/3945061351 conn(0x7f0c6006abe0 msgr2=0x7f0c600c1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:58.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 shutdown_connections 2024-01-20T08:59:58.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.782+0000 7f0c65f8d640 1 -- 172.21.15.87:0/3945061351 wait complete. 2024-01-20T08:59:58.794 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-01-20T08:59:58.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.902+0000 7f7fd37d1640 1 Processor -- start 2024-01-20T08:59:58.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.902+0000 7f7fd37d1640 1 -- start start 2024-01-20T08:59:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.903+0000 7f7fd37d1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.903+0000 7f7fd37d1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7fcc15ac70 0x7f7fcc0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.903+0000 7f7fd37d1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fcc0ca6b0 con 0x7f7fcc0c9090 2024-01-20T08:59:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.903+0000 7f7fd37d1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fcc0cc7a0 con 0x7f7fcc15ac70 2024-01-20T08:59:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.903+0000 7f7fd37d1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0caee0 con 0x7f7fcc0c8cb0 2024-01-20T08:59:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.904+0000 7f7fd156d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.904+0000 7f7fd1d6e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7fcc15ac70 0x7f7fcc0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.904+0000 7f7fd156d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc0cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:41254/0 (socket says 172.21.15.87:41254) 2024-01-20T08:59:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.904+0000 7f7fd156d640 1 -- 172.21.15.87:0/1492657386 learned_addr learned my addr 172.21.15.87:0/1492657386 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:58.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.906+0000 7f7fd156d640 1 -- 172.21.15.87:0/1492657386 >> v2:172.21.15.116:3300/0 conn(0x7f7fcc15ac70 msgr2=0x7f7fcc0cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-20T08:59:58.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.906+0000 7f7fd156d640 1 --2- 172.21.15.87:0/1492657386 >> v2:172.21.15.116:3300/0 conn(0x7f7fcc15ac70 0x7f7fcc0cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.906+0000 7f7fd156d640 1 -- 172.21.15.87:0/1492657386 >> v1:172.21.15.116:6789/0 conn(0x7f7fcc0c9090 legacy=0x7f7fcc0c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.906+0000 7f7fd156d640 1 -- 172.21.15.87:0/1492657386 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc0cb1c0 con 0x7f7fcc0c8cb0 2024-01-20T08:59:58.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.909+0000 7f7fd156d640 1 --2- 172.21.15.87:0/1492657386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc0cc260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009e50 tx=0x7f7fc402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f96009f43c94d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fc3fff640 1 -- 172.21.15.87:0/1492657386 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc402cb90 con 0x7f7fcc0c8cb0 2024-01-20T08:59:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fc3fff640 1 -- 172.21.15.87:0/1492657386 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7fc402ccf0 con 0x7f7fcc0c8cb0 2024-01-20T08:59:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fc3fff640 1 -- 172.21.15.87:0/1492657386 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4034630 con 0x7f7fcc0c8cb0 2024-01-20T08:59:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 -- 172.21.15.87:0/1492657386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 msgr2=0x7f7fcc0cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/1492657386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc0cc260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009e50 tx=0x7f7fc402bff0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 -- 172.21.15.87:0/1492657386 shutdown_connections 2024-01-20T08:59:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/1492657386 >> v2:172.21.15.116:3300/0 conn(0x7f7fcc15ac70 0x7f7fcc0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/1492657386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc0cc260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 -- 172.21.15.87:0/1492657386 >> 172.21.15.87:0/1492657386 conn(0x7f7fcc08af90 msgr2=0x7f7fcc08b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 -- 172.21.15.87:0/1492657386 shutdown_connections 2024-01-20T08:59:58.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.913+0000 7f7fd37d1640 1 -- 172.21.15.87:0/1492657386 wait complete. 2024-01-20T08:59:58.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 Processor -- start 2024-01-20T08:59:58.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 -- start start 2024-01-20T08:59:58.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fcc0c9090 0x7f7fcc152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fcc15ac70 0x7f7fcc0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0c5390 con 0x7f7fcc0c9090 2024-01-20T08:59:58.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0cfd50 con 0x7f7fcc15ac70 2024-01-20T08:59:58.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.915+0000 7f7fd37d1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0cee90 con 0x7f7fcc0c8cb0 2024-01-20T08:59:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.916+0000 7f7fd156d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.916+0000 7f7fd1d6e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fcc15ac70 0x7f7fcc0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.916+0000 7f7fd156d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.87:41262/0 (socket says 172.21.15.87:41262) 2024-01-20T08:59:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.916+0000 7f7fd156d640 1 -- 172.21.15.87:0/2737780056 learned_addr learned my addr 172.21.15.87:0/2737780056 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-01-20T08:59:58.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.921+0000 7f7fd1d6e640 1 -- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 msgr2=0x7f7fcc152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:58.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.921+0000 7f7fd1d6e640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.921+0000 7f7fd1d6e640 1 -- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fcc0c9090 msgr2=0x7f7fcc152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:58.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.921+0000 7f7fd1d6e640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fcc0c9090 0x7f7fcc152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:58.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.921+0000 7f7fd1d6e640 1 -- 172.21.15.87:0/2737780056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009b30 con 0x7f7fcc15ac70 2024-01-20T08:59:58.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.921+0000 7f7fd156d640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-20T08:59:58.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.924+0000 7f7fd1d6e640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fcc15ac70 0x7f7fcc0cdad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800e7f0 tx=0x7f7fc800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:58.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.927+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800c730 con 0x7f7fcc15ac70 2024-01-20T08:59:58.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.927+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7fc800c890 con 0x7f7fcc15ac70 2024-01-20T08:59:58.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.927+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800cca0 con 0x7f7fcc15ac70 2024-01-20T08:59:58.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.927+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc0cf3a0 con 0x7f7fcc15ac70 2024-01-20T08:59:58.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.928+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc0cf790 con 0x7f7fcc15ac70 2024-01-20T08:59:58.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.929+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc074290 con 0x7f7fcc15ac70 2024-01-20T08:59:58.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.949+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7fc8004020 con 0x7f7fcc15ac70 2024-01-20T08:59:58.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.949+0000 7f7fc1ffb640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7fb006af00 0x7f7fb006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-20T08:59:58.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.949+0000 7f7fd156d640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7fb006af00 0x7f7fb006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-20T08:59:58.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.949+0000 7f7fd156d640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7fb006af00 0x7f7fb006d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c090 tx=0x7f7fc4030000 comp rx=0 tx=0).ready entity=mgr.4154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-20T08:59:58.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:58.952+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f7fc80581d0 con 0x7f7fcc15ac70 2024-01-20T08:59:59.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.194+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7fc8054510 con 0x7f7fcc15ac70 2024-01-20T08:59:59.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.364+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7fcc0c8690 con 0x7f7fcc15ac70 2024-01-20T08:59:59.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.969+0000 7f7fc1ffb640 1 -- 172.21.15.87:0/2737780056 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f7fc8021020 con 0x7f7fcc15ac70 2024-01-20T08:59:59.971 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data_ec' removed 2024-01-20T08:59:59.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7fb006af00 msgr2=0x7f7fb006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:59.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7fb006af00 0x7f7fb006d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c090 tx=0x7f7fc4030000 comp rx=0 tx=0).stop 2024-01-20T08:59:59.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fcc15ac70 msgr2=0x7f7fcc0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fcc15ac70 0x7f7fcc0cdad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800e7f0 tx=0x7f7fc800ecb0 comp rx=0 tx=0).stop 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 shutdown_connections 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:6832/2055452307,v1:172.21.15.87:6833/2055452307] conn(0x7f7fb006af00 0x7f7fb006d3c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fcc15ac70 0x7f7fcc0cdad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fcc0c9090 0x7f7fcc152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 --2- 172.21.15.87:0/2737780056 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fcc0c8cb0 0x7f7fcc152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.978+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 >> 172.21.15.87:0/2737780056 conn(0x7f7fcc08af90 msgr2=0x7f7fcc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-20T08:59:59.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.979+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 shutdown_connections 2024-01-20T08:59:59.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-01-20T08:59:59.980+0000 7f7fd37d1640 1 -- 172.21.15.87:0/2737780056 wait complete. 2024-01-20T08:59:59.994 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-20T08:59:59.995 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-20T08:59:59.995 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T08:59:59.995 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T08:59:59.995 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i a" 2024-01-20T08:59:59.996 INFO:tasks.ceph.mds.a:Stopped 2024-01-20T08:59:59.996 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-20T08:59:59.996 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T08:59:59.996 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T08:59:59.996 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i c" 2024-01-20T08:59:59.996 INFO:tasks.ceph.mds.c:Stopped 2024-01-20T08:59:59.996 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-01-20T08:59:59.997 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T08:59:59.997 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T08:59:59.997 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i e" 2024-01-20T08:59:59.997 INFO:tasks.ceph.mds.e:Stopped 2024-01-20T08:59:59.997 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-20T08:59:59.997 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:00.010 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-01-20T09:00:00.000+0000 b5ce640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i b (PID: 41983) UID: 0 2024-01-20T09:00:00.011 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-01-20T09:00:00.002+0000 b5ce640 -1 mds.b *** got signal Terminated *** 2024-01-20T09:00:03.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~6s 2024-01-20T09:00:03.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~6s 2024-01-20T09:00:03.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-20T09:00:03.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-20T09:00:03.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2024-01-20T09:00:06.099 INFO:tasks.ceph.mds.b:Stopped 2024-01-20T09:00:06.100 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-20T09:00:06.100 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T09:00:06.100 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi116 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i d" 2024-01-20T09:00:06.100 INFO:tasks.ceph.mds.d:Stopped 2024-01-20T09:00:06.100 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-01-20T09:00:06.100 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:06.101 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T09:00:06.101 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi116 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i f" 2024-01-20T09:00:06.101 INFO:tasks.ceph.mds.f:Stopped 2024-01-20T09:00:06.101 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-20T09:00:06.101 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-20T09:00:06.101 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:06.102 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-01-20T09:00:06.102+0000 7f74c31c9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 0 (PID: 36201) UID: 0 2024-01-20T09:00:06.102 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-01-20T09:00:06.102+0000 7f74c31c9640 -1 osd.0 32 *** Got signal Terminated *** 2024-01-20T09:00:09.584 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-01-20T09:00:09.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-01-20T09:00:09.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2024-01-20T09:00:12.203 INFO:tasks.ceph.osd.0:Stopped 2024-01-20T09:00:12.203 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-20T09:00:12.204 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:12.204 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T09:00:12.204+0000 7f5e8bc2e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 1 (PID: 36215) UID: 0 2024-01-20T09:00:12.205 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-01-20T09:00:12.204+0000 7f5e8bc2e640 -1 osd.1 33 *** Got signal Terminated *** 2024-01-20T09:00:15.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-01-20T09:00:15.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-01-20T09:00:16.722 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:00:16.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:00:17.171 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:30:01.650 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.174 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:30:01.652 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.177 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:30:01.613 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.178 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:30:01.614 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.180 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:30:01.616 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.180 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:30:01.616 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.261 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:30:01.740 35945== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.272 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:30:01.708 35919== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:17.273 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:30:01.709 35918== Warning: unimplemented fcntl command: 1036 2024-01-20T09:00:18.305 INFO:tasks.ceph.osd.1:Stopped 2024-01-20T09:00:18.305 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-20T09:00:18.305 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:18.306 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-01-20T09:00:18.306+0000 7fd28b73f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 2 (PID: 36214) UID: 0 2024-01-20T09:00:18.306 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-01-20T09:00:18.306+0000 7fd28b73f640 -1 osd.2 35 *** Got signal Terminated *** 2024-01-20T09:00:21.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-01-20T09:00:21.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-01-20T09:00:24.407 INFO:tasks.ceph.osd.2:Stopped 2024-01-20T09:00:24.407 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-20T09:00:24.407 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:24.408 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-01-20T09:00:24.407+0000 7fdc6d962640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 3 (PID: 36220) UID: 0 2024-01-20T09:00:24.408 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-01-20T09:00:24.407+0000 7fdc6d962640 -1 osd.3 36 *** Got signal Terminated *** 2024-01-20T09:00:27.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-01-20T09:00:27.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-01-20T09:00:30.509 INFO:tasks.ceph.osd.3:Stopped 2024-01-20T09:00:30.509 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-20T09:00:30.510 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:30.510 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-01-20T09:00:30.510+0000 7f3afd3c9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 4 (PID: 36495) UID: 0 2024-01-20T09:00:30.510 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-01-20T09:00:30.510+0000 7f3afd3c9640 -1 osd.4 37 *** Got signal Terminated *** 2024-01-20T09:00:33.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-01-20T09:00:33.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-01-20T09:00:36.610 INFO:tasks.ceph.osd.4:Stopped 2024-01-20T09:00:36.611 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-20T09:00:36.611 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:36.612 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-01-20T09:00:36.611+0000 7f2712036640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 5 (PID: 36492) UID: 0 2024-01-20T09:00:36.612 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-01-20T09:00:36.611+0000 7f2712036640 -1 osd.5 38 *** Got signal Terminated *** 2024-01-20T09:00:39.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-01-20T09:00:39.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-01-20T09:00:42.712 INFO:tasks.ceph.osd.5:Stopped 2024-01-20T09:00:42.712 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-20T09:00:42.712 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:42.713 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-01-20T09:00:42.713+0000 7fc3e5bab640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 6 (PID: 36494) UID: 0 2024-01-20T09:00:42.713 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-01-20T09:00:42.713+0000 7fc3e5bab640 -1 osd.6 39 *** Got signal Terminated *** 2024-01-20T09:00:44.727 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-01-20T09:00:44.727 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-01-20T09:00:46.823 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:00:46.826 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:00:48.814 INFO:tasks.ceph.osd.6:Stopped 2024-01-20T09:00:48.815 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-20T09:00:48.815 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:48.816 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-01-20T09:00:48.815+0000 7f8eec64d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 7 (PID: 36498) UID: 0 2024-01-20T09:00:48.816 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-01-20T09:00:48.815+0000 7f8eec64d640 -1 osd.7 39 *** Got signal Terminated *** 2024-01-20T09:00:50.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-01-20T09:00:50.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-01-20T09:00:54.916 INFO:tasks.ceph.osd.7:Stopped 2024-01-20T09:00:54.917 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-20T09:00:54.917 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-20T09:00:54.917 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:54.918 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T09:00:54.917+0000 7f1d62f72640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 35959) UID: 0 2024-01-20T09:00:54.918 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-01-20T09:00:54.917+0000 7f1d62f72640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-01-20T09:00:54.945 INFO:tasks.ceph.mgr.y:Stopped 2024-01-20T09:00:54.945 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-20T09:00:54.945 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:54.970 INFO:tasks.ceph.mgr.x:Stopped 2024-01-20T09:00:54.970 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-20T09:00:54.970 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-20T09:00:54.970 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:00:55.001 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-01-20T09:00:54.988+0000 9f78640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 35922) UID: 0 2024-01-20T09:00:55.002 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-01-20T09:00:54.990+0000 9f78640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-01-20T09:00:55.495 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:30:39.974 35945== 2024-01-20T09:00:55.495 INFO:tasks.ceph.mon.a.smithi087.stderr:==00:00:30:39.974 35945== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T09:00:55.507 INFO:tasks.ceph.mon.a.smithi087.stderr:daemon-helper: command failed with exit status 42 2024-01-20T09:00:55.536 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-01-20T09:01:00.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-01-20T09:01:01.072 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T09:01:01.072 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2024-01-20T09:01:01.073 INFO:tasks.ceph.mon.a:Stopped 2024-01-20T09:01:01.073 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-20T09:01:01.073 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:01:01.089 INFO:tasks.ceph.mon.b.smithi116.stderr:2024-01-20T09:01:01.077+0000 9f78640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 35911) UID: 0 2024-01-20T09:01:01.089 INFO:tasks.ceph.mon.b.smithi116.stderr:2024-01-20T09:01:01.078+0000 9f78640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-01-20T09:01:01.642 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:30:46.077 35918== 2024-01-20T09:01:01.642 INFO:tasks.ceph.mon.b.smithi116.stderr:==00:00:30:46.077 35918== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T09:01:01.652 INFO:tasks.ceph.mon.b.smithi116.stderr:daemon-helper: command failed with exit status 42 2024-01-20T09:01:05.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-01-20T09:01:05.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-01-20T09:01:07.174 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T09:01:07.175 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi116 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2024-01-20T09:01:07.175 INFO:tasks.ceph.mon.b:Stopped 2024-01-20T09:01:07.175 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-20T09:01:07.175 INFO:teuthology.orchestra.run:waiting for 300 2024-01-20T09:01:07.191 INFO:tasks.ceph.mon.c.smithi116.stderr:2024-01-20T09:01:07.179+0000 9f78640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 35914) UID: 0 2024-01-20T09:01:07.191 INFO:tasks.ceph.mon.c.smithi116.stderr:2024-01-20T09:01:07.181+0000 9f78640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2024-01-20T09:01:10.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-01-20T09:01:17.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:01:17.158 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:01:18.196 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:31:02.632 35919== 2024-01-20T09:01:18.196 INFO:tasks.ceph.mon.c.smithi116.stderr:==00:00:31:02.632 35919== Exit program on first error (--exit-on-first-error=yes) 2024-01-20T09:01:18.207 INFO:tasks.ceph.mon.c.smithi116.stderr:daemon-helper: command failed with exit status 42 2024-01-20T09:01:19.377 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-20T09:01:19.377 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi116 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2024-01-20T09:01:19.378 INFO:tasks.ceph.mon.c:Stopped 2024-01-20T09:01:19.378 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-20T09:01:19.378 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v MON_DOWN | head -n 1 2024-01-20T09:01:19.479 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T09:01:19.479 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-20T09:01:19.559 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T09:01:19.559 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-20T09:01:19.648 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T09:01:19.648 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-20T09:01:19.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T09:01:19.697 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-20T09:01:19.807 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T09:01:19.807 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-20T09:01:19.922 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T09:01:19.922 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-20T09:01:20.019 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T09:01:20.019 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-20T09:01:20.075 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T09:01:20.075 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-20T09:01:20.126 INFO:tasks.ceph:Archiving mon data... 2024-01-20T09:01:20.131 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/data/mon.a.tgz 2024-01-20T09:01:20.131 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-01-20T09:01:20.147 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.bXgmuxvJyI 2024-01-20T09:01:20.148 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.bXgmuxvJyI 2024-01-20T09:01:20.283 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.bXgmuxvJyI 2024-01-20T09:01:20.368 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.bXgmuxvJyI is 338KB 2024-01-20T09:01:20.464 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.bXgmuxvJyI 2024-01-20T09:01:20.479 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/data/mon.b.tgz 2024-01-20T09:01:20.479 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-01-20T09:01:20.496 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.j4cqCNmbXC 2024-01-20T09:01:20.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.j4cqCNmbXC 2024-01-20T09:01:20.650 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.j4cqCNmbXC 2024-01-20T09:01:20.734 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.j4cqCNmbXC is 343KB 2024-01-20T09:01:20.829 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.j4cqCNmbXC 2024-01-20T09:01:20.843 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/data/mon.c.tgz 2024-01-20T09:01:20.844 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-01-20T09:01:20.897 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.0aMKU6kCjc 2024-01-20T09:01:20.898 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0aMKU6kCjc 2024-01-20T09:01:21.023 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.0aMKU6kCjc 2024-01-20T09:01:21.108 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.0aMKU6kCjc is 343KB 2024-01-20T09:01:21.176 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.0aMKU6kCjc 2024-01-20T09:01:21.191 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-20T09:01:21.191 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-20T09:01:21.193 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-20T09:01:21.485 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable malloc malloc strdup 2024-01-20T09:01:21.752 INFO:tasks.ceph:Archiving crash dumps... 2024-01-20T09:01:21.754 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/remote/smithi087/crash 2024-01-20T09:01:21.755 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-20T09:01:21.783 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/remote/smithi116/crash 2024-01-20T09:01:21.784 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-20T09:01:21.810 INFO:tasks.ceph:Shutting down logrotate 2024-01-20T09:01:21.811 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:01:21.828 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-20T09:01:21.910 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-20T09:01:21.938 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-20T09:01:21.965 INFO:tasks.ceph:Compressing logs... 2024-01-20T09:01:21.965 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-20T09:01:21.983 DEBUG:teuthology.orchestra.run.smithi116:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-20T09:01:22.062 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-20T09:01:22.062 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2024-01-20T09:01:22.062 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.f.log 2024-01-20T09:01:22.062 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.b.log 2024-01-20T09:01:22.062 INFO:teuthology.orchestra.run.smithi116.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2024-01-20T09:01:22.062 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/valgrind/mds.f.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 67.7% -- replaced with /var/log/ceph/valgrind/mds.f.log.gz 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.d.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/valgrind/mds.b.log: /var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 57.3% -- replaced with /var/log/ceph/valgrind/mds.b.log.gz 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/valgrind/mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 67.8% -- replaced with /var/log/ceph/valgrind/mds.d.log.gz 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36150.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36270.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36150.log.gz 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36310.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36270.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 93.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-20T09:01:22.063 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36270.log.gz 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36310.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36310.log.gz 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36106.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36190.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36106.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36106.log.gz 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36390.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mgr.x.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36190.log.gz 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36390.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36390.log.gz 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-01-20T09:01:22.064 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mon.b.log: 93.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-20T09:01:22.065 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-20T09:01:22.065 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36230.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-01-20T09:01:22.065 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36350.log 2024-01-20T09:01:22.068 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.d.log: 94.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-20T09:01:22.068 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-20T09:01:22.068 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36350.log: /var/log/ceph/ceph-mon.c.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36350.log.gz 2024-01-20T09:01:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49056.log 2024-01-20T09:01:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2024-01-20T09:01:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49056.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-20T09:01:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-01-20T09:01:22.076 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49056.log.gz 2024-01-20T09:01:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mon.a.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gz 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42615.log 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41816.log 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42615.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2%/var/log/ceph/ceph-client.admin.41816.log: -- replaced with /var/log/ceph/ceph-client.admin.42615.log.gz 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49588.log 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.41732.log: gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41816.log.gz -- 2024-01-20T09:01:22.077 INFO:teuthology.orchestra.run.smithi087.stderr: /var/log/ceph/ceph-client.admin.46158.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42531.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49588.log.gz 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46158.log.gz 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43508.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42531.log.gz 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50324.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48830.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-01-20T09:01:22.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44064.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50324.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49798.log 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45232.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.c.log 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49798.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.a.log 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/valgrind/mds.c.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49798.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/valgrind/client.0.log 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr: 67.7% -- replaced with /var/log/ceph/valgrind/mds.c.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/valgrind/mds.a.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr: 67.7% -- replaced with /var/log/ceph/valgrind/mds.a.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/valgrind/client.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.e.log 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr: 76.9% -- replaced with /var/log/ceph/valgrind/client.0.log.gz 2024-01-20T09:01:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35961.log 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/valgrind/mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45948.log 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr: 67.7% -- replaced with /var/log/ceph/valgrind/mds.e.log.gz 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49924.log 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43550.log 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49924.log: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.35961.log.gz 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45948.log.gz 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49924.log.gz 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46634.log 2024-01-20T09:01:22.080 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43550.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45990.log 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46634.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50508.log 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46634.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49140.log 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45990.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45413.log 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49140.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50508.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44840.log 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6%/var/log/ceph/ceph-client.admin.45413.log: -- replaced with /var/log/ceph/ceph-client.admin.49140.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50384.log 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-01-20T09:01:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50718.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44840.log.gz 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50550.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50384.log.gz 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43650.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42447.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50718.log.gz 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43650.log.gz 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42489.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 2024-01-20T09:01:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42489.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44654.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4%/var/log/ceph/ceph-client.admin.44386.log: -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50466.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46200.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44654.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50466.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41774.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50466.log.gz 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.41999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2024-01-20T09:01:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.41774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42209.log 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49630.log: gzip 88.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41774.log.gz -- 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr: /var/log/ceph/ceph-client.admin.46032.log 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8%/var/log/ceph/ceph-client.admin.42209.log: -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42209.log.gz 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45906.log 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46374.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46032.log.gz 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45006.log 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45822.log 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45006.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45906.log.gz 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-01-20T09:01:22.084 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50092.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45006.log.gz 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6%/var/log/ceph/ceph-client.admin.50092.log: -- replaced with /var/log/ceph/ceph-client.admin.45822.log.gz 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43424.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49966.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50092.log.gz 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43734.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43424.log.gz 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49966.log.gz 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43734.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44958.log 88.2% 2024-01-20T09:01:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43734.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.audit.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44958.log.gz/var/log/ceph/ceph-client.admin.44645.log: 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45696.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49224.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.41957.log: /var/log/ceph/ceph-client.admin.45696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44022.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45696.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47020.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44022.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49224.log.gz 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42083.log 2024-01-20T09:01:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44022.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.47020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50592.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50592.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44435.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50592.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46242.log: /var/log/ceph/ceph-client.admin.44435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44190.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44435.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50218.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42573.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44190.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50218.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44190.log.gz 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45024.log 2024-01-20T09:01:22.087 INFO:teuthology.orchestra.run.smithi087.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50218.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42573.log: /var/log/ceph/ceph-client.admin.45024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45612.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42573.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.1.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45024.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44659.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45612.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42272.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44659.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9%/var/log/ceph/ceph-client.admin.43938.log: -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48788.log 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 2024-01-20T09:01:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49545.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48788.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42272.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45570.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48788.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49545.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49840.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45570.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49545.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50760.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49840.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45570.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43592.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49840.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43592.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.50760.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr: /var/log/ceph/ceph-client.admin.45738.log 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43592.log.gz 2024-01-20T09:01:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46074.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45738.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45738.log.gz/var/log/ceph/ceph-client.admin.46074.log: 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50424.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46074.log.gz 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48998.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42363.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50424.log: /var/log/ceph/ceph-client.admin.48998.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50424.log.gz 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44653.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42363.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48998.log.gz 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49756.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42363.log.gz 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42319.log 2024-01-20T09:01:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49756.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44653.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49098.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49756.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49098.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42319.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50260.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43382.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49098.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36362.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50260.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36362.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.45654.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50260.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3%/var/log/ceph/ceph-mds.e.log: -- replaced with /var/log/ceph/ceph-client.admin.36362.log.gz 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48956.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45654.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-20T09:01:22.091 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1%/var/log/ceph/ceph-client.admin.48956.log: -- replaced with /var/log/ceph/ceph-client.admin.45654.log.gz 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48956.log.gz 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41913.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.45864.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43980.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.41913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44657.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41913.log.gz 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43980.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43980.log.gz 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45486.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41486.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44657.log.gz/var/log/ceph/ceph-client.admin.45486.log: 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44660.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45486.log.gz/var/log/ceph/ceph-client.admin.41486.log: 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-01-20T09:01:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41486.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44886.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44660.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36225.log 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42167.log: /var/log/ceph/ceph-client.admin.36225.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36225.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44646.log 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46308.log 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44646.log: /var/log/ceph/ceph-client.admin.41572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44646.log.gz 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50008.log 2024-01-20T09:01:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49882.log 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46308.log: /var/log/ceph/ceph-client.admin.50008.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50008.log.gz 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46308.log.gz 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45192.log 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43818.log 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49882.log.gz 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45192.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43818.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45192.log.gz 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50134.log 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43818.log.gz 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2024-01-20T09:01:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50134.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50134.log.gz 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49672.log 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.41687.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49672.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49672.log.gz 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43466.log 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48914.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50050.log 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43466.log.gz 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50050.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50050.log.gz 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43692.log 2024-01-20T09:01:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.log: 87.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43692.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43692.log.gz 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44649.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-20T09:01:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44477.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2024-01-20T09:01:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44258.log 2024-01-20T09:01:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45780.log 2024-01-20T09:01:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44258.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44258.log.gz 2024-01-20T09:01:22.108 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49714.log 2024-01-20T09:01:22.108 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45780.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45780.log.gz 2024-01-20T09:01:22.112 INFO:teuthology.orchestra.run.smithi116.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-01-20T09:01:22.122 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46116.log 2024-01-20T09:01:22.123 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49714.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49714.log.gz 2024-01-20T09:01:22.126 INFO:teuthology.orchestra.run.smithi116.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-20T09:01:22.131 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45302.log 2024-01-20T09:01:22.131 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46116.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46116.log.gz 2024-01-20T09:01:22.141 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 2024-01-20T09:01:22.141 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45302.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45302.log.gz 2024-01-20T09:01:22.155 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42657.log 2024-01-20T09:01:22.155 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44300.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-01-20T09:01:22.158 INFO:teuthology.orchestra.run.smithi087.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-01-20T09:01:22.171 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44342.log 2024-01-20T09:01:22.171 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42657.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.42657.log.gz 2024-01-20T09:01:22.184 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50176.log 2024-01-20T09:01:22.184 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44342.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44342.log.gz 2024-01-20T09:01:22.195 INFO:teuthology.orchestra.run.smithi087.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-20T09:01:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-01-20T09:01:22.196 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50176.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50176.log.gz 2024-01-20T09:01:22.196 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45083.log 2024-01-20T09:01:22.206 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46506.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz 2024-01-20T09:01:22.206 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29933.log 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45083.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45083.log.gz 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49450.log 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.tmp-client.admin.29933.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29933.log.gz 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45340.log 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49450.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49450.log.gz 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44106.log 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45340.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45340.log.gz 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42125.log 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44106.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44106.log.gz 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43776.log 2024-01-20T09:01:22.207 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42125.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42125.log.gz 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48872.log 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43776.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44878.log 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48872.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48872.log.gz 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44878.log.gz 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-01-20T09:01:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42041.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz 2024-01-20T09:01:22.210 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-20T09:01:22.210 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43157.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-01-20T09:01:22.210 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50676.log 2024-01-20T09:01:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42723.log 2024-01-20T09:01:22.232 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50676.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-01-20T09:01:22.232 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50676.log.gz 2024-01-20T09:01:22.243 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.42723.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45528.log 2024-01-20T09:01:22.243 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42723.log.gz 2024-01-20T09:01:22.243 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50634.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-01-20T09:01:22.257 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45528.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45528.log.gz 2024-01-20T09:01:22.317 INFO:teuthology.orchestra.run.smithi087.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-20T09:01:22.633 INFO:teuthology.orchestra.run.smithi116.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-20T09:01:22.685 INFO:teuthology.orchestra.run.smithi116.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-20T09:01:23.359 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-20T09:01:24.749 INFO:teuthology.orchestra.run.smithi087.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-20T09:01:24.844 INFO:teuthology.orchestra.run.smithi116.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-20T09:01:25.381 INFO:teuthology.orchestra.run.smithi116.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-20T09:01:25.412 INFO:teuthology.orchestra.run.smithi116.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-20T09:01:25.421 INFO:teuthology.orchestra.run.smithi116.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-20T09:01:25.423 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-01-20T09:01:25.423 INFO:teuthology.orchestra.run.smithi116.stderr:real 0m3.398s 2024-01-20T09:01:25.423 INFO:teuthology.orchestra.run.smithi116.stderr:user 0m13.841s 2024-01-20T09:01:25.423 INFO:teuthology.orchestra.run.smithi116.stderr:sys 0m0.456s 2024-01-20T09:01:25.998 INFO:teuthology.orchestra.run.smithi087.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-20T09:01:26.095 INFO:teuthology.orchestra.run.smithi087.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-20T09:01:26.302 INFO:teuthology.orchestra.run.smithi087.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-20T09:01:34.783 INFO:teuthology.orchestra.run.smithi087.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.47020.log.gz 2024-01-20T09:01:41.770 INFO:teuthology.orchestra.run.smithi087.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-20T09:01:41.772 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-01-20T09:01:41.772 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m19.779s 2024-01-20T09:01:41.772 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m47.555s 2024-01-20T09:01:41.772 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m1.594s 2024-01-20T09:01:41.772 INFO:tasks.ceph:Archiving logs... 2024-01-20T09:01:41.772 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/remote/smithi087/log 2024-01-20T09:01:41.773 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-20T09:01:49.021 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/remote/smithi116/log 2024-01-20T09:01:49.022 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-20T09:01:51.250 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc strdup 2024-01-20T09:01:51.252 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-20T09:01:51.261 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_a00c9940991c5e1e9ab595faa3f6e1281d8ae167/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc strdup 2024-01-20T09:01:51.261 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-20T09:01:51.262 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-20T09:01:51.264 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-20T09:01:51.346 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-20T09:01:51.357 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-20T09:01:51.357 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-20T09:01:51.360 DEBUG:teuthology.orchestra.run.smithi116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-20T09:01:51.374 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-01-20T09:01:51.376 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-01-20T09:01:51.379 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-20T09:01:51.379 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-01-20T09:01:51.379 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 65 -705us[ -737us] +/- 65ms 2024-01-20T09:01:51.379 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv02.front.sepia.ceph.com 3 6 377 60 -1961us[-1993us] +/- 66ms 2024-01-20T09:01:51.379 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +2713us[+2682us] +/- 69ms 2024-01-20T09:01:51.379 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T09:01:51.380 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-20T09:01:51.380 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-01-20T09:01:51.381 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 61 -599us[ -602us] +/- 65ms 2024-01-20T09:01:51.381 INFO:teuthology.orchestra.run.smithi116.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -1818us[-1820us] +/- 66ms 2024-01-20T09:01:51.381 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 126 +2888us[+2851us] +/- 68ms 2024-01-20T09:01:51.381 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-20T09:01:51.381 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-20T09:01:51.390 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-20T09:01:51.391 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-20T09:01:51.401 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-20T09:01:51.495 DEBUG:teuthology.orchestra.run.smithi116:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-20T09:01:51.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-20T09:01:51.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T09:01:51.596 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-20T09:01:51.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T09:01:51.629 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-20T09:01:51.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-20T09:01:51.647 INFO:teuthology.task.internal:Duration was 2632.381821 seconds 2024-01-20T09:01:51.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-20T09:01:51.655 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-20T09:01:51.655 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-20T09:01:51.658 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-20T09:01:51.699 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-20T09:01:51.712 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-20T09:01:51.984 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-20T09:01:51.984 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-01-20T09:01:51.984 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-20T09:01:52.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi116.front.sepia.ceph.com 2024-01-20T09:01:52.044 DEBUG:teuthology.orchestra.run.smithi116:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-20T09:01:52.067 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-20T09:01:52.067 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-20T09:01:52.087 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-20T09:01:52.135 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-20T09:01:52.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-20T09:01:52.155 DEBUG:teuthology.orchestra.run.smithi116:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-20T09:01:52.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-20T09:01:52.292 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-20T09:01:52.293 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-20T09:01:52.319 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-20T09:01:52.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-20T09:01:52.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-20T09:01:52.369 DEBUG:teuthology.orchestra.run.smithi116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-20T09:01:52.392 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-01-20T09:01:52.413 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = core 2024-01-20T09:01:52.444 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-20T09:01:52.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T09:01:52.463 DEBUG:teuthology.orchestra.run.smithi116:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-20T09:01:52.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-20T09:01:52.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-20T09:01:52.505 INFO:teuthology.task.internal:Transferring archived files... 2024-01-20T09:01:52.505 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/remote/smithi087 2024-01-20T09:01:52.506 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-20T09:01:52.541 DEBUG:teuthology.misc:Transferring archived files from smithi116:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433/remote/smithi116 2024-01-20T09:01:52.541 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-20T09:01:52.576 INFO:teuthology.task.internal:Removing archive directory... 2024-01-20T09:01:52.577 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-20T09:01:52.579 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-20T09:01:52.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-20T09:01:52.637 INFO:teuthology.task.internal:Not uploading archives. 2024-01-20T09:01:52.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-20T09:01:52.647 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-20T09:01:52.647 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-20T09:01:52.651 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-20T09:01:52.663 INFO:teuthology.orchestra.run.smithi087.stdout: 265944 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 20 09:01 /home/ubuntu/cephtest 2024-01-20T09:01:52.681 INFO:teuthology.orchestra.run.smithi116.stdout: 265291 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 20 09:01 /home/ubuntu/cephtest 2024-01-20T09:01:52.682 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-20T09:01:52.699 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-20T09:01:52.711 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: Leak_StillReachable malloc malloc strdup 2024-01-20T09:01:52.711 INFO:teuthology.nuke:Checking targets against current locks 2024-01-20T09:01:52.740 DEBUG:teuthology.nuke:shortname: smithi087 2024-01-20T09:01:52.740 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-20T09:01:52.765 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-20 08:09:25.502947', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGB32BTEDO0f4g3p4Su/WnQlZgOmLrecAJhuA4qWv3Hx1cNDmUiiybUgT27c7A5Yy4fY0ICXZmnZzUdRNPO3Oio='} 2024-01-20T09:01:52.769 DEBUG:teuthology.nuke:shortname: smithi116 2024-01-20T09:01:52.769 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-20T09:01:52.792 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-19_15:24:44-fs-main-distro-default-smithi/7522433', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-20 08:09:25.501498', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE16rhgvJXoLMH67byLqrLa9Gu1eNFmFGCvaBgtCERHmUQ1R4dGsq0qngj1/LXWBVgR6VFpcqXa3CROxwDuOR1k='} 2024-01-20T09:01:52.820 INFO:teuthology.orchestra.console.smithi087:Power off 2024-01-20T09:01:52.820 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-20T09:01:52.842 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2024-01-20T09:01:52.842 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:01:52.859 INFO:teuthology.orchestra.console.smithi116:Power off 2024-01-20T09:01:52.859 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-20T09:01:52.868 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-01-20T09:01:52.881 DEBUG:teuthology.orchestra.console.smithi116:power off output: Chassis Power Control: Down/Off 2024-01-20T09:01:52.882 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:01:52.923 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-01-20T09:01:56.869 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:01:56.924 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:01:56.980 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-01-20T09:01:57.035 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-01-20T09:02:00.985 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:02:01.036 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:02:01.095 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-01-20T09:02:01.148 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-01-20T09:02:05.096 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:02:05.149 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-20T09:02:05.207 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2024-01-20T09:02:05.260 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2024-01-20T09:02:05.308 INFO:teuthology.orchestra.console.smithi087:Power off completed 2024-01-20T09:02:05.361 INFO:teuthology.orchestra.console.smithi116:Power off completed 2024-01-20T09:02:05.466 INFO:teuthology.run:Summary data: description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} duration: 2632.381820678711 failure_reason: 'valgrind error: Leak_StillReachable malloc malloc strdup' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-01-20T09:02:05.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-20T09:02:05.566 INFO:teuthology.run:FAIL